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ABSTRACT 


Title  of  Dissertation:  Discrete  Representation  of  Signals  from  Infinite 

Dimensional  Hilbert  Spaces  with  Application  to 
Noise  Suppression  and  Compression 

Anthony  Teolis,  Doctor  of  Philosophy,  1993 

Dissertation  directed  by:  Professor  J.  J.  Benedetto 

Department  of  Mathematics 
and  Professor  S.  A.  Shamma 

Department  of  Electrical  Engineering 


Addressed  in  this  thesis  is  the  issue  of  representing  signals  from  infinite  dimensional  Hilbert 
spaces  in  a  discrete  form.  The  discrete  representations  which  are  studied  come  from  the  irregular 
samples  of  a  signal  dependent  transform  called  the  group  representation  transform,  e.g.,  the 
wavelet  and  Gabor  transforms.  The  main  issues  dealt  with  are  (i)  the  recoverability  of  a  signal 
from  its  discrete  representation,  (ii)  the  suppression  of  noise  in  a  corrupted  signal,  and  (iii) 
compression  through  efficient  discrete  representation. 

The  starting  point  of  the  analysis  lies  with  the  intimate  connection  between  the  DufRn- 
Schaeffer  theory  of  (global)  frames  and  irregular  sampling  theory.  This  connection  has  lead 
elsewhere  to  the  formulation  of  iterative  schemes  for  the  reconstruction  of  a  signed  from  its 
irregular  samples.  However,  these  schemes  have  not  addressed  such  issues  as  digital  imple- 
mentability  and  reconstruction  from  perturbed  representations.  Here,  iterative  reconstruction 
algorithms  are  developed  and  implemented  which  recover  a  signal  from  its  possibly  perturbed 
discrete  representation. 

Robustness  to  perturbations  occurring  directly  in  the  signal  domain  are  also  investigated. 
Based  on  the  notion  of  coherence  with  respect  to  a  frame,  a  simple  non-linear  thresholding 
scheme  is  developed  for  the  rejection  of  noise. 

The  structure  of  the  discretization  has  many  free  parameters  including  the  choice  of  group 
representation  transform,  the  analyzing  function  associated  with  the  group  representation  trans¬ 
form,  and  the  sampling  set.  Each  choice  of  parameters  leads  to  a  different  discrete  representation 
and  the  specification  of  an  underlying  set  of  primitive  functions.  Reconstructability  is  directly 
related  to  the  frame  properties  of  this  set  of  primitive  functions. 

Localized  discrete  representations  around  a  particular  signal  are  also  investigated.  Trunca¬ 
tions  and  other  signal  dependent  localization  of  global  representations  lead  to  finite  representa¬ 
tions.  The  approach  to  finite  representations  which  is  taken  here  can  be  stated  in  terms  of  local 
frames  for  the  reproducing  kernel  Hilbert  space  formed  by  the  range  of  the  group  representation 
transform. 

Finally,  numerical  examples  of  discrete  representations  which  are  signal  independent  and 
new  signal  dependent  discrete  (positive  extrema)  wavelet  representations  are  presented.  Recon¬ 
struction,  noise  suppression,  and  compression  experiments  are  conducted  and  demonstrated  on 
numerical  examples  including  speech  and  synthetic  signals. 
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Chapter  1 

Introduction 


In  this  dissertation  we  develop  theory  and  implement  algorithms  regarding,  first,  the  discrete 
representation  of  signals  belonging  to  certain  infinite  dimensional  Hilbert  spaces,  and  second, 
the  reconstruction  of  these  signals  from  their  discrete  representations.  Our  formulation  of  the 
problem  leads  to  applications  including  signal  compression  and  noise  suppression. 


1.1  Motivation 


Many  infinite  dimensional  (Hilbert)  spaces  are  full  of  redundancy  in  the  sense  that  they 
consist  of  elements  (signals)  which  admit  complete  discrete  representations.  That  is,  such 
spaces  consist  of  signals  which  admit  discrete  representations  which  uniquely  describe  them. 
Let  us  call  such  spaces  as  compressible.  The  canonical  example  of  compressible  spaces  are  the 
Paley- Wiener  spaces  made  up  of  all  functions  band-limited  by  a  certain  fixed  constant.  They 
are  functional  Hilbert  spaces  consisting  of  elements  whose  Fourier  transforms  have  compact 
support. 

Many  signals  which  occur  in  nature  may  be  considered  to  come  from  infinite  dimensional 
spaces.  For  example,  the  variation  in  air  pressure  caused  by  a  sound  source  or  the  intensity 
of  electromagnetic  energy  reflected  by  an  object  illuminated  by  an  active  source,  e.g.,  sunlight. 
Sound  and  sight  are  just  two  examples  of  natural  mechanisms  which  contain  and  convey  infor¬ 
mation.  Because  the  mediums  in  which  these  signals  exist  are  inherently  analog  their  direct 
processing  by  digital  means  requires  discretization. 

Digital  techniques  have  fast  become  the  prefered  technology  in  a  wide  range  of  practical 
applications.  Perhaps  the  best  example  of  this  comes  from  the  voice  communication  industry. 
For  transmission  of  spoken  messages,  the  superiority  of  digitally  coded  analog  voice  over  straight 
analog  transmission  has  been  firmly  established  [Gib93].  Of  the  many  benefits  associated  with 
digital  coding  the  two  foremost  are  low  susceptability  to  noise  in  communication  channels  and 
the  ability  of  manipulation  of  the  data  by  digital  computers.  With  communication  constraints 
such  as  fixed  capacity  and  limited  bandwidth  channels,  high  information  content  signals  can 
not  be  transmitted  directly  over  existing  channels.  A  theory  of  discrete  representations  which 
addresses  issues  such  as  compression  and  reconstructions  is  directly  applicable  to  this  problem. 

With  the  pervasiveness  of  digital  processing  which  permeates  modern  engineering  and  indus¬ 
try,  the  fundamental  understanding  of  discrete  representations  is  of  great  practical  importance. 
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1.2  Perspective 


The  most  natural  example  of  a  discrete  signal  representation  in  an  infinite  dimensional 
Hilbert  space  is  one  where  residing  signals  are  evaluated  at  a  discrete  set  of  points,  i.e.  sampling. 
As  such  the  theory  of  discrete  representations  and  the  theory  of  sampling  are  intimately  related. 

Uniform  sampling  on  the  space  of  band-limited  functions,  also  called  the  Paley- Wiener 
space1,  offers  a  well-known  example  of  a  discrete  representation.  Known  to  Whittaker  [Whil5] 
in  1915,  and  applied  later  in  1933  by  Kotel’nikov  [Kot33]  to  communication  problems,  the 
so  called  “Sampling  Theorem”  did  not  find  fame  until  the  1949  landmark  paper  of  Shannon 
[Sha49].  Recall  that  the  Classical  Sampling  Theorem  establishes  that  any  function  /,  in  the 
space  of  functions  band-limited  by  fl,  can  be  represented  in  terms  of  its  samples  as 


/«  =  T'£f(nT) 


sin(27rO(a:  -  nT )) 
tt(x  —  nT) 


(where  the  sum  converges  in  L 2)  provided  that  2TCI  <  1.  Note  that  in  the  case  2 TO  =  1  the 
set  of  functions  {  }  1S  an  orthonormal  basis  for  the  space  of  functions  band-limited 

by  Cl. 

Many  of  the  developments  in  what  we  will  call  “discretization  theory”  can  be  viewed  as 
generalizations  to  the  Classical  Sampling  Theorem.  In  particular,  one  direction  of  generalization 
is  to  remove  the  restriction  that  the  sampling  sequence  be  regular.  Marvasti  [Mar87]  offers 
an  extensive,  but  dated,  bibliography  on  problems  related  to  irregular  sampling.  Irregular 
sampling  theory  for  band-limited  spaces  has  been  studied  extensively  by  Paley- Wiener  [PW34], 
Levinson  [Lev40],  Beutler  [Beu61],  [Beu66],  and  Yao-Thomas  [YT67].  And  recently  all  of  the 
afore-mentioned  theory  has  been  extended  and  unified  by  Benedetto-Heller  [BII90],[Hel91],  and 
Benedetto  [Ben92]  in  the  context  of  frames.  Frames  were  originally  introduced  by  Buflin  and 
Schaefer  [DS52]  for  use  in  problems  related  to  non-harmonic  Fourier  series.  The  theory  of 
frames  plays  a  fundamental  role  in  irregular  sampling  and  discretization  theory. 

Developing  sampling  theorems  for  spaces  other  than  those  that  are  band-limited  is  an  addi¬ 
tional  level  of  generalization  to  the  Classical  Sampling  Theorem.  Two  types  of  spaces  that  are 
of  particular  interest  are  generated  as  the  image  of  (i)  the  Gabor  transform  and  (ii)  the  (affine) 
wavelet  transform.  The  continuous  wavelet  and  Gabor  transforms  are  reviewed  in  Chapter  2. 
The  first  space  has  its  roots  with  Gabor  [Gab46]  and  the  coherent  states  of  quantum  physics 
while  the  second,  has  seen  recent  and  significant  growth  with  Meyer  and  Daubechies  [Mey90] 
[Dau]  [DGM86]  [Dau90j.  Both  of  these  transforms  can  be  identified  with  groups:  in  the  case 
of  the  Gabor  transform  the  Weyl- Heisenberg  group  of  modulations  and  translations  and  in  the 
case  of  the  wavelet  transform  the  affine  or  ax  +  b  group  of  dilations  and  translations. 

Both  the  wavelet  transform  and  the  Gabor  transform  have  associated  discretizations  and 
sampling  theorems.  To  see  this,  consider  the  case  of  the  wavelet  transform.  An  affine  wavelet 
transform  is  a  continuous  transformation  that  maps,  for  example,  L2(IR),  the  space  of  finite  en¬ 
ergy  signals  of  one  real  variable,  to  finite  energy  signals  on  the  affine  or  ax  +  b  group.  Sampling 
the  wavelet  transform  “regularly”  (with  respect  to  the  group  structure),  one  obtains  the  (reg¬ 
ular)  discrete  wavelet  transform.  Theorems  for  the  reconstruction  of  a  signal  from  its  discrete 


1  Because  of  a  result  of  Paley  and  Wiener  which  characterizes  the  band-limited  functions  as  entire  functions 
of  exponential  type,  such  band-limited  spaces  are  labeled  as  Paley-Wiener  spaces. 
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wavelet  transform  are  sampling  theorems;  and  the  regularly  sampled  discrete  wavelet  transform 
of  a  signal  /  of  finite  energy  is  a  discrete  representation  of  /. 

Irregular  sampling  in  spaces  associated  with  the  Gabor  transform  and  the  wavelet  transform 
has  been  addressed  by  Grochenig  [Gro92].  Introduced  by  Mallat  [MZ92b]  in  the  context  of 
image  processing,  the  wavelet  maxima  representation  can  be  thought  of  as  a  specific  irregular 
sampling  of  the  continuous  wavelet  transform.  In  this  work  Mallat  has  employed  the  method 
of  projection  onto  convex  sets  [YW91]  to  obtain  reconstructions. 

Another  space  of  interest  is  that  of  the  Bargmann-Fock  space.  Irregular  sampling  in  the 
Bargmann-Fock  spaces  has  been  investigated  by  Seip  [Sei92a],[Sei92b].  Ogawa  [Oga89]  has 
taken  an  operator  theoretic  approach  to  the  sampling  problem  at  the  level  of  Reproducing 
Kernel  Hilbert  Spaces  (RKHS).  Venturing  further,  out  of  the  realm  of  Hilbert  spaces,  into 
the  realm  of  Banach  spaces,  Grochenig  and  Feichtinger  [FG89]  provide  an  extensive  theory  of 
“atomic  decompositions”  as  discrete  representations.  Here,  inner  product  dependent  properties 
are  replaced  by  analogous  convolution  formulations.  In  this  thesis  we  shall  limit  the  scope  of 
the  discussion  to  discretization  in  the  more  practical  Hilbert  spaces  and  not  consider  sampling 
in  general  Banach  spaces. 

1.3  Objectives 


As  mentioned  earlier  our  primary  goal  is  to  develop  both  theory  and  numerical  algorithms 
for  the  representation  and  reconstruction  of  signals  from  (compressible)  Hilbert  spaces.  In 
particular,  the  following  are  some  of  the  major  objectives  of  this  research: 

(i)  To  develop  tools  for  the  study  of  invertible  and  stable  discrete  representations  of  signals 
resident  in  certain  Hilbert  spaces  (in  particular,  reproducing  kernel  Hilbert  spaces  (RKHS) 
)• 

(ii)  To  provide  implementable  methods  for  the  reconstruction  of  these  signals  from  their 
discrete  representations  and  to  provide  bounds  for  the  associated  reconstruction  error. 

(iii)  To  develop  efficient  methods  for  determining  finite  representations  from  discrete  represen¬ 
tations  and  to  provide  bounds  on  the  associated  error  in  reconstructions  due  to  truncation. 

(iv)  To  identify  properties  of  discrete  representations  which  make  them  suitable  for  practical 
applications,  e.g.,  robustness  to  noise  and  coefficient  imprecision. 

(v)  To  provide  particular  representations;  for  example,  the  wavelet  extrema  representation  of 
Mallat  [MZ92b], 

(vi)  To  apply  the  theoretical  analysis  developed  and  to  implement  the  associated  algorithms 
in  the  particular  signal  processing  applications  of  data  compression  and  noise  suppression. 

1.4  Approach 


This  research  has  two  distinct  components.  First,  it  has  an  analytic  or  theoretical  compo¬ 
nent  and,  second,  it  has  a  computational  or  implementational  component.  Analytically  it  is 
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desirable  to  work  at  a  level  of  sufficient  generality  as  to  capture  the  essential  features  of  the 
subject.  Computationally,  it  is  desirable  to  draw  algorithms  from  a  theory  which  is  directly 
implementable.  We  wish  to  find  an  intermediate  ground  in  which  a  coherent  and  implementable 
theory  can  be  successfully  developed.  Because  of  these  issues  we  believe  that  a  proper  setting 
for  our  investigation  is  in  the  Reproducing  Kernel  Hilbert  Spaces. 

On  the  computational  side,  we  are  interested  in  developing  and  implementing  algorithms 
for  the  reconstruction  of  signals  taken  from  certain  infinite  dimensional  Hilbert  spaces.  The 
reconstruction  should  use  only  knowledge  of  a  particular  signal’s  discrete  representation.  Our 
approach  to  this  problem  will  involve  reproducing  kernel  Hilbert  spaces  in  a  fundamental  way. 
The  two  main  examples  of  RKHS’s  which  we  will  target  numerically  are  the  Paley- Wiener  space 
and  the  image  of  wavelet  transforms. 

On  the  analysis  side,  the  RKHS’s  of  interest  will  be  infinite  dimensional  (separable)  com¬ 
pressible  Hilbert  spaces.  This  space  is  denoted  H.  Discrete  representations  of  elements  in  this 
space  will  reside  in  an  infinite  dimensional  (discrete)  space  which  has  functions  with  support 
on  a  discrete  lattice.  This  space  is  denoted  (?  (Z).  Finally,  finite  discrete  representations  of 
elements  in  'H  will  be  derived  as  finite  truncations  of  corresponding  elements  in  i2  (Z).  This 
finite  dimensional  (discrete)  space  which  has  functions  with  support  on  a  discrete  lattice  is 
denoted  as  Hf. 

We  have  thus  associated  with  the  three  spaces  H,  t2  (Z)  and  Hf  the  notions  of  compressible, 
discrete,  and  finite  representations,  respectively.  Now,  we  define  the  mappings  L  (discretization) 
and  F  (truncation)  through  the  statement 

Hkt2(7L)£>  Hf. 

The  research  described  here  can  be  thought  of  succinctly  as  a  detailed  study  of  the  properties 
of  the  two  operators  L  and  F. 

For  instance,  suppose  L  is  an  invertible  bounded  linear  operator  so  that  L  is  a  topological 
isomorphism.  Consequently,  the  two  spaces  H  and  L(7i)  are  topologically  equivalent.  Therefore 
there  is  no  loss  of  information  in  the  discrete  representation.  Thus,  with  respect  to  information 
preserving  discretizations,  there  is  no  need  to  require  the  image  L{7i)  of  the  discretization 
operator  L,  to  be  the  whole  space  l2  (Z).  2  Hence,  this  freedom  may  be  exploited  to  achieve 
certain  goals,  e.g.,  noise  suppression  and  signal  compression. 

Example  1.4.1  identifies  a  discretization  operator  for  the  familiar  case  of  uniform  sampling 
in  a  Paley- Wiener  space.  We  see  that  sampling  rates  strictly  above  the  Nyquist  rate  imply  the 
injectivity  of  the  discretization  operator;  and  that  rates  at  the  Nyquist  rate  further  imply  the 
surjectivity  of  the  discretization  operator. 

We  denote  the  Fourier  transform  of  a  signal  /  as  /  and  formally  define  it  as 

Ki)  =  J 

where  i  =  1-  This  definition  is  valid  on  T1(1R)  and  can  be  extended  to  functions  in  T2(]R), 

viz.  Chapter  2. 

2This  situation  is  to  be  contrasted  with  the  problem  of  interpolation  in  which  one  starts  out  with  a  repre¬ 
sentation  in  F  (Z)  and  asks  to  find  an  /  in  H  which  has  this  representation.  For  interpolation  problems  it  is 
required  that  L  be  surjective. 
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Example  1.4.1  (Uniform  Sampling  in  PC  =  PWq)  Given  ft  >  0,  the  Paley-  Wiener  space, 
PWq,  is  defined  as 


PWq  =  {/£X2(]R):supp/C[-ft,ft]}. 


It  is  a  RKHS  with  kernel 


Kx(y)  = 


sin(27rft(y  —  x )) 
7T  (y-x) 


and  reproducing  formula 


V/GPWn,  f(x)  =  {f,Kx). 


For  2Tft  <  1,  we  have  the  well-known  Whittaker-Kotelnikov-Shannnon  formula 


V/  €  PWq, 


/(*)  =  r£/(»T) 


sin(27rft(x  —  nT )) 
n(x  —  nT) 


which  may  be  rewritten  as 


VfePWa ,  f  =  T'£<f,KnT>KnT. 


Hence  for  the  case  PC  =  PWq  we  may  define  the  discretization  operator  L  :  PWq  i2  (71.)  as 
the  mapping 

Lf  =  {<f,VTKnT>} 
with  the  adjoint  L*  :  i2  (Z)  t->  PWq 

L*C  =  ^CnVfKnT. 

This  L  is  an  injective  isomorphism  when  2 Tft  <  1  and  is  surjective  when  2 Tft  =  1.  Thus,  for 
the  case  2Tft  =  1,  L  is  bijective. 


1.4.1  Discrete  Representations  and  Frames 

Let  PC  denote  a  separable  Hilbert  space.  The  main  tool  which  we  employ  in  our  investigation 
is  the  theory  of  (Hilbert  space)  frames  due  to  Duffin  and  Schaeffer  [DS52].  As  we  will  subse¬ 
quently  show,  the  power  of  frame  theory  lies  in  the  fact  that  the  associated  “frame  operator” 
S  has  many  important  properties;  these  include  the  following: 

(i)  S  has  the  factorization  S  -  L*L,  e.g.,  [DGM86],  [Dau90],  where  L  is  a  “discretization” 
operator  which  associates  with  any  f  £  PL  the  discrete  representation  Lf  ( L *  is  its  adjoint); 
and 

(ii)  S  is  invertible. 

These  two  properties  are  key  elements  of  our  analysis.  To  begin  with,  we  have 

V/€7f,  f  =  S~1L*Lf.  (1.4.1) 

In  this  equation  we  see  that  any  signal  /  can  be  fully  represented  by  the  discrete  vector  Lf; 
i.e.  the  original  signal  /  can  be  recovered  from  the  discrete  representation  Lf.  In  other  words 
/  and  Lf  contain  the  same  information. 
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1.4.2  Finite  Representations 

Through  frame  theory  we  are  able  to  achieve  countable  representations  of  signals  resident 
in  any  separable  Hilbert  space;  however,  we  are  ultimately  interested  in  the  finite  dimensional 
representations  of  signals.  It  is  evident  that  a  method  to  go  from  the  countable  to  the  finite 
is  desired.  We  have  previously  introduced  such  a  transformation  as  the  mapping  F.  In  this 
section  we  discuss  some  of  the  pertinent  issues  concerning  the  operator  F. 

We  have  seen  that  any  signal  in  a  compressible  Hilbert  space  H  has  a  representation  in 
terms  of  a  countable  set  of  coefficients.  However,  we  would  like  to  represent  signals  in  H  in 
terms  of  a  finite  set  of  coefficients.  This  leads  naturally  to  the  following  questions: 

(i)  How  is  a  finite  set  of  coefficients  chosen? 

(ii)  What  are  bounds  on  the  associated  approximation  error? 

Here,  approximation  error  refers  to  the  error  incurred  by  performing  reconstruction  based  on 
the  finite  set  of  coefficients.  To  answer  these  questions  we  identify  and  analyze  the  properties 
of  an  appropriate  operator,  F. 

In  order  to  “truncate”  representations  in  £ 2  (Z)  so  that  the  resulting  truncations  are  mem¬ 
bers  of  a  finite  dimensional  Hilbert  space  Hf,  we  introduce  the  truncation  operator,  F  :  t2  (Z)  i-> 
Hf  C  (?  (Z).  We  think  of  Hf  as  the  representation  space.  The  best  representation  space  for  a 
given  signal  f*€H  will,  of  course,  depend  largely  on  the  signal  itself.  Therefore  the  truncation 
operator  F  (respectively,  the  space  Hf)  will  depend  on  a  specific  /*  EH  and  an  error  tolerance 
6  >  0.  When  pertinent  we  will  write  Ff,  or  Fj,ts  (respectively,  Hj{f *)  or  Hf{f*,6))  to  make 
the  dependence  explicit.  The  representation  space  Hj  can  then  be  characterized  as 

^/(/*)  —  Ff,L(H), 

the  image  of  the  composite  operator  Ff,L.  The  signal  dependence  of  the  representation  space 
is  of  great  interest  in  applications  such  as  signal  compression  and  noise  suppression.  Along 
with  the  fact  that  FL  is  a  continuous  mapping,  the  fact  that  the  representation  space  is  signal 
dependent  suggests  that  it  may  be  useful  as  describing  signals  which  are  alike,  i.e.  classification 
based  on  representation. 


1.5  Synopsis 


Our  methods  are  demonstrated  on  numerical  examples  including  speech  and  synthetic  sig¬ 
nals. 

In  chapter  2  we  present  background  material. 

In  chapter  3  we  review  the  theory  of  frames  and  develop  some  useful  theoretical  tools. 

In  chapter  4  we  develop  discrete  representations  in  a  general  setting  of  reproducing  kernel 
Hilbert  spaces  and  give  specific  examples  of  such  spaces  and  their  representation. 

In  chapter  5  we  introduce  the  notion  of  a  local  frame  and  its  relationship  to  various  signal 
dependent  representations. 

In  chapter  6  we  provide  numerical  examples  of  discrete  signal  representations  and  their 
reconstructions.  In  addition,  we  apply  our  discrete  representation  theory  to  the  areas  of  signal 
compression  and  noise  suppression.  In  particular  we  demonstrate  our  approach  on  speech 
signals. 
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Chapter  2 

Mathematical  Preliminaries 


This  chapter  introduces  the  basic  notation,  concepts,  and  tools  which  we  shall  employ 
throughout  this  thesis.  In  particular,  we  introduce  the  pertinent  basic  spaces  with  which  we 
shall  deal,  the  Fourier  transform  and  some  important  basic  theorems  which  are  associated  with 
it.  In  addition  we  present  some  concepts  from  group  theory  which  are  pertinent  to  the  discrete 
representation  of  analog  signals.  Also  we  describe  the  continuous  versions  of  the  Gabor  and 
Wavelet  transforms.  Finally  some  notions  from  sampling  theory  are  reviewed.  Most  of  the 
basic  notation  and  concepts  which  we  use  may  be  found  in  standard  texts  on  real  analysis  and 
operator  theory,  e.g.,  [Roy68]  and  [GG80].  Material  dealing  with  groups  and  weighted  spaces 
is  in  the  same  vain  as  [FG89]  and  [HW89]  and  much  of  the  material  on  sampling  may  be  found 
in  [You80]  or  [Ben92]. 

2.1  Notation  and  Tools 


2.1.1  Basic  Symbols  and  Functions 

Z  denotes  the  integers. 

1R  denotes  the  real  numbers. 

IR+  =  {t  G  ]R  :  t  >  0}  denotes  the  strictly  positive  real  numbers. 

IR~  =  {t  E  JR  :  t  <  0}  denotes  the  strictly  negative  real  numbers. 

(C  denotes  the  complex  numbers  and  i  =  \/— 1. 

The  complex  conjugate  of  z  6  (C  is  denoted  z  and  the  magnitude  of  2  is  denoted  \z\. 

T  =  {z  €  (C  :  \z\  =  1}  is  the  torus  of  complex  numbers  with  unit  magnitude,  i.e.  the  unit 
circle  in  the  complex  plane. 

If  5  is  a  set  then  the  characteristic  function  Is  of  the  set  S  is  defined  as 


!sO)  = 


i,  x  e  s, 

0,  otherwise. 


The  function  Sm  n  is  the  Kronecker  delta  function  defined  as 


j  1,  m  =  n, 
1  0,  m  ^  n. 
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The  Dirichlet  kernel  or  sine  function  d  is  defined  as 

,,  .  sin(f) 

and  for  a  constant  A  >  0  the  L 1 II. III.  norm  preserving  dilation  d\  of  d  is 

dx(t)  =  Xd(Xt). 

2.1.2  Basic  Function  and  Operator  Spaces 

An  essential  requirement  of  the  Hilbert  spaces  which  are  to  be  discretized  is  that  they  admit 
dense  countable  subsets,  i.e.  that  they  be  separable.  Because  of  this  whenever  the  discussion 
entails  an  arbitrary  Hilbert  space  that  Hilbert  space  should  be  assumed  separable.  No  further 
mention  of  the  separability  issue  will  be  made. 

The  three  fundamental  Hilbert  spaces  with  which  we  deal  are  L2(1R),  ^2(2),  and  PWq. 
Also  of  interest  is  the  Banach  space  of  all  linear  bounded  operators  mapping  one  Hilbert  space 
to  another. 

T2(IR)  is  the  space  of  complex-valued  finite  energy  signals  defined  on  the  real  line  IR.  The 
norm  of  an  element  /  £  L2(1R)  is 

ll/ll  =  (y  l/0)|2<y) 2  <  00, 

where  integration  is  over  51,  and  the  inner  product  of  f,g  £  L2(]R)  is  {/,  g)  =  f  f{t)g[t)dt. 

S?  (2)  is  the  space  of  complex- valued  finite  energy  sequences  defined  on  the  integers  2.  The 
norm  of  an  element  c  £  (?  (2)  is 


cll  =  (2  lcn|2)  2  <  oo, 


where  summation  is  over  2,  and  the  inner  product  of  c,  d  £  l2  (2)  is  (c,d)  =  ^  cndn. 

is  the  space  of  bounded  linear  operators  which  map  the  Hilbert  space  ft\  to  the 
Hilbert  space  7^2-  The  norm  of  an  element  K  £  B{fi\ , H.2)  is 


||T||  =  sup 

x£H\ 


1 \Txbu 

Ml*. 


<  00. 


2.1.3  Bases  and  Completeness  in  Hilbert  Space 

Let  Ti  be  a  separable  Hilbert  space  and  be  a  sequence  of  elements  in  H.  denotes 

the  closure. 


I.  span{^>„}  is  the  set  of  vectors  generated  as  linear  combinations  of  the  elements  of  {4>n}, 
i.e.  span  {<£„}  =  {£n=i  cn<j>n  :  cn  £  (C,  N  £  2}. 

II.  {4>n}  is  dense  in  H  if  span  {4>n}  =  ft. 

III.  {4>n}  is  complete  in  ft  if  (/,  <f>n)  =  0  if  and  only  if  /  =  0. 
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IV.  {4>n}  is  orthonormal  in  H  if  (<j>m,4>n) 

V.  {<l>n}  is  a  Schauder  basis  or  basis  for  H  if  for  each  /  £  Ft  there  is  a  unique  sequence 
{cn}  £  (B  such  that  /  =  J2cn<t>n- 

VI.  An  orthonormal  set  {<j>n}  is  an  orthonormal  basis  for  H  if  for  every  f  £  Ft  there  is  sequence 
{cn}  £  (D  such  that  /  =  Yicn<t>n-  If  {<^n}  is  an  orthonormal  basis  then  for  every  f  £  Ft 

/  =  £</,^n 
71 

and 

ll/ll2  =  £  I  {/><M  |2  (Parseval’s  equality.) 

71 

VII.  A  basis  {4>n}  is  a  Riesz  basis  for  Ft  if  it  is  related  to  an  orthonormal  basis  by  a  topological 
isomorphism,  i.e.  there  is  a  topological  isomorphism  T  :Ft  i-*  Ft  such  that  <f>n  =  Ten  for 
all  n  where  {en}  is  an  orthonormal  basis  for  Ft. 

VIII.  A  basis  {</>„}  is  an  unconditional  basis  for  Ti  if  every  convergent  series  of  the  form  J2  cn<l>n 
is  unconditionally  convergent,  i.e.  every  arrangement  of  its  terms  converges  to  the  same 
element. 

IX.  An  unconditional  basis  {4>n}  is  a  bounded  unconditional  basis  for  Ft  if  there  are  constants 
0  <  A  <  B  <  oo  such  that  A  <  ||<^n||  <  B. 

X.  If  {ipn}  is  another  sequence  in  H  then  {</»„}  and  {ipn}  are  biorthogonal  if  (4>m,ipn)  =  <5TOi„. 

2.1.4  Fourier  Transform 

The  Fourier  transform  is  a  mapping  T  :  X2(1R)  h-*  X2(ffi,)  as  follows.  For  /  £  Xx(lR)  C  X2(1R) 

?f{l)  =  hi)  =  /  f(t)e~2^dt,  (2.1.1) 

and  for  /  g  X2(IR)  \  XX(]R) 

ff(l)  =  Rl)  =  lim  r  me-^dt,  (2.1.2) 

n— kx>  J_n 

for  7  £  IR(=  ]R).  Convergence  of  the  integrals  to  T f  =  /  is  in  the  X2-sense. 

We  shall  almost  exclusively  adopt  the  notation  to  indicate  the  Fourier  transform. 

The  Fourier  transform  of  f  £  X2(1R)  may  be  represented  as 

f(l)  =  (f  5  c7)l2(R)  (2.1.3) 

and  consequently  the  inverse  transform  as 

/W  =  </.«-.)I?(fi).  (2.1.4) 

Two  fundamental  results  in  Fourier  theory  are  the  Plancherel  and  Parseval  relations.  Let 

f,geL2(] R). 
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Plancherel 


/  \mi2dt  =  J \fM\2d7. 


Parseval 


J  f{t)9(t)dt  =  J  f{'r)g{'r)d'y. 


The  Plancherel  and  Parseval  relations  may  be  represented  as  ||/||  =  ||/||  and  (/, g)z,2CR)  — 

( />^)L2(rj  resPectively-  In  this  form  the  relations  also  hold  for  spaces  other  than  Z.2(1R).  The 

Fourier  transform  is  a  unitary  map  from  L2(EL)  to  L2(jt). 

Two  further  spaces  of  interest  are  the  Hardy  spaces  H^_  and  if2  given  as 


and 


H+  =  {/  6  L\U)  :  supp  /  C  [0,oo)j 
Hi  =  {f  £  i2(IR)  :  supp  f  C  (-00, 0] | , 


where  supp  /  is  the  support  of  /.  Let  f+  be  an  arbitrary  element  of  77?  and  /_  an  arbitrary 
element  of  7/2 .  The  Hardy  space  //2  is  a  Hilbert  space  with  norm 


II/+IIH5  =  (jT°  I/+(t)I! 

with  the  inner  product  it  induces.  The  Hardy  space  H 2  is  a  Hilbert  space  with  norm 

||/_ll,e  =  (/_J/-(7)I: 


with  the  inner  product  it  induces.  Also  Hi  and  H2  are  orthogonal  complements  in  L2(IR), 
i.e.,  Z/2(]R)  =  Hi  ®  II2  .  Consequently,  for  an  arbitrary  element  /  in  L2(IR,)  there  are  elements 
/_  6  Hi  and  f+  £  Hi  so  that  /  =  /_  +  /+  and  ||/||2  =  \\f-\\2H2  +  ||/+||^2  • 


Remark  2.1.1  Any  real  element  of  L2(]R)  may  be  associated  with  either  i/2  or  Hi  since 
all  real  functions  have  Fourier  transforms  which  are  involutive  (conjugate  symmetric),  i.e.,  if 

/  is  real  then  /( 7)  =  /(— 7).  Thus,  real  functions  are  uniquely  determined  by  their  Fourier 
transforms  on  [0,oo)  or  (— 00, 0]. 


2.1.5  Operators 

Let  7i\  and  H2  be  arbitrary  Hilbert  spaces  with  norms  ||.||i  and  and  inner  products 
(•,  -)j  and  (•,  -)2  respectively.  Let  T  :  Ti\  »-»•  H2- 

I.  The  range  of  T  is  T(Hi)  =  {Tf  :  f  £  Hi}. 

II.  The  kernel  of  T  is  kerT  =  {/  £  Hi  :  Tf  =  0). 
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III.  T  is  injective  or  one-to-one  if  Tf  =  Tg  if  and  only  if  /  =  g.  If  T  is  a  linear  operator  then 
T  is  injective  if  and  only  if  ker  T  =  {0}. 

IV.  T  is  surjective  or  onto  if  T(H\)  —  Hi- 

V.  T  is  bijective  if  it  is  both  injective  and  surjective. 

VI.  T  has  an  inverse  T~1  :  Hi  »-*■  Hi  if  T  is  bijective.  In  this  case  the  inverse  of  T  is  defined 
as  T-1#  =  /  if  g  =  Tf. 

VII.  T  is  continuous  if  xn  — »  x  implies  Txn  — >  Tx.  A  linear  operator  T  is  bounded  if  and  only 
if  T  is  continuous. 

VIII.  The  adjoint  of  T  is  the  unique  operator  T*  :  H 2  Hi  so  that  (Tf,g)2  =  ( f,T*g)1  for 

all  /  6  Hi  and  g  €  7^2-  T  is  self  adjoint  if  T*  =  T. 

IX.  T  e  B{fH\,H<i)  is  a  compact  operator  if  for  all  sequences  {fn  :  ||/n||  =  1}  C  Hi  the 
sequence  {Txn}  has  a  converging  subsequence  in  Hi- 

X.  T  is  a  topological  isomorphism  if  T  is  bijective,  T  6  B(H\,Hi)  and  T_1  6  B(Hi,H\). 
Thus,  both  T  and  T-1  are  continuous  linear  mappings. 

XI.  T  is  an  isometry  if  for  all  /  e  Hi,  ||T/||2  =  ||/||i. 

XII.  T  is  a  unitary  map  if  it  is  linear,  bijective,  and  an  isometry.  If  T  is  unitary  then  T_1  =  T*. 

Compositions  of  topological  isomorphisms  are  again  topological  isomorphisms.  Two  Hilbert 
spaces  H\  and  Hi  which  are  related  by  a  topological  isomorphism  are  topologically  equivalent. 
Topologically  equivalent  spaces  are  in  an  abstract  sense  the  same.  A  topological  isomorphism 
which  is  norm  preserving  is  a  unitary  map.  Compositions  of  isometries  are  again  isometries. 

With  H'  a  subspace  of  a  Hilbert  space  H  the  operator  :  H  H'  is  the  orthogonal 
projection  operator  onto  H' . 

Fact  2.1.2  ([GG80,  Theorem  II. 13.1])  P  E  B(H,H)  is  an  orthogonal  projection  if  and  only 
if  P2  =  P  and  P  is  self  adjoint. 


Fact  2.1.3  ([GG80,  Theorem  III.4.1])  If  T  E  B(H,H)  is  self  adjoint  then 


||T||  =  sup 
fen 


\(f,Tf)\ 


Let  /  be  an  arbitrary  element  of  L2(1R.)  and  take  a,  b  and  s  >  0  all  to  be  real  numbers. 
I.  The  involution  operator  ~ :  L2(IR)  L2(IR)  is  a  unitary  map  given  by 

m  =  7 (-«)- 
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II.  The  translation  operator  ra  :  T2(1R)  »-»  i2(IR)  is  a  unitary  map  given  by 

(raf)(t)  = 

III.  The  modulation  operator  eb  :  Z2(1R)  Z2(1R)  is  a  unitary  map  given  by 

(ebf)(t)  ^  e2*ibtf(t). 

IV.  The  dilation  operator  Ds  :  £2(1R)  t-v  T2(IR,)  is  a  unitary  map  given  by 

(Dsf)(t)  =  s12f(st). 

Table  2.1  displays  in  a  compact  form  these  unitary  operators  on  L2(1R),  their  definitions,  in¬ 
verses,  and  Fourier  transforms. 

Of  special  interest  are  the  composite  relations: 

ean  =  e2*tabTbea  and  rbDa  =  Darab. 


Name 

Uf 

(u/m 

i 

II 

* 

(UfT 

translation 

modulation 

dilation 

involution 

Taf 

eaf 

Dsf 

I 

f(t  -  a) 
e2wiatf(t) 
sif(st) 

7  i-t) 

T-af 

e-af 

D,-if 

1 

e-af 

Taf 

D.-if 

■x 

/ 

Table  2.1:  Unitary  Operators  U  :  T2(1R)  i-»  Z/2(1R),  their  inverses  (equal  to  their  adjoints),  and 
their  Fourier  transforms.  /  e  T2(IR),  a, b  E  1R  and  s  G  1R+. 


The  convolution  f  *  g  of  two  functions  f,g  E  T2(1R)  is 

(. f*g)(t)  =  J  f(x)g(t-x)dx. 

Convolution  may  also  be  represented  as 

(f*9)(t)  =  (f,Ttg)  ■ 

By  standard  arguments  (/  *  g)'  —  fg.  To  see  this  explicitly  write 

/  *  d(t)  =  (/,  rtg) 

=  (J,  e_tg^  (by  Parseval) 

=  {fg,e-t)  =  (fd)v(t) 
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2.1.6  Bandlimited  Spaces 

Paley  and  Wiener  [PW34]  have  made  significant  fundamental  contributions  to  Fourier  theory 
and,  in  particular,  the  understanding  of  bandlimited  functions.  For  this  reason  we  label  spaces 
of  bandlimited  functions  as  “Paley-Wiener”  spaces.  Bandlimited  functions  are  defined  as  those 
which  have  compact  support  in  the  frequency  domain,  i.e.  their  Fourier  transforms  vanish  off 
some  compact  set.  One  result  of  Paley  and  Wiener  characterizes  any  0  bandlimited  space  as 
one  consisting  of  entire  functions  of  exponential  type  2 nO. 

For  0  >  0  PWq  is  the  Paley-  Wiener  space  defined  as 

PWq  =  {/6i2(l R)  :  supp  /  C  [-«,«]}. 

Recall  that  a  complex  function  /  :  ( C  i — »■  <C  is  analytic  at  a  point  zq  £  (C  if  the  complex 
derivative  f'(z )  exists  for  all  points  in  some  open  neighborhood  containing  20- 

Definition  2.1.4  (Entire  functions  of  exponential  type)  Let  /  :  (C  > — »  (C . 

a.  /  is  an  entire  function  if  it  is  analytic  on  all  of  (C . 

b.  An  entire  function  /  is  of  exponential  type  A  if 

Vz<E(C,  \f(z)\<MeBW 

for  some  positive  constant  M  =  M(B)  and  all  B  >  A. 


Theorem  2.1.5  ([PW34,  Theorem  X])  A  function  is  (l  bandlimited  if  and  only  if  it  is  an 
entire  function  of  exponential  type  2x0.  i.e., 

PWq  =  {/  :  /  is  an  entire  function  of  exponential  type  2 xO}  . 


For  a  <  b  £  IR  PW[a^  is  the  Paley-Wiener  space  defined  as 

PW[a,b]  =  {feL\l R)  :  supp  /C  [«,&]}. 

For  0,2  >  >  0,  PWqX'Q2  is  the  Paley-Wiener  space  defined  as 

PWq i,n2  =  {/  €  L2(1R)  :  supp  /  C  u  [Oi,fl2]}  , 

Clearly  PWQiiq2  =  PW{_q2  _ni]  ©  PW[Quq2]. 

L2[a,b]  is  the  space  of  finite  energy  signals  defined  on  the  interval  [a,  b].  The  Fourier 
transform  is  an  isomorphism  between  the  spaces  PW[o  f)]  and  L2  [a,  b]  and  hence,  they  are  topo¬ 
logically  equivalent.  In  particular,  the  spaces  L 2  [ — 0,  S7]  and  PWq  are  topologically  equivalent 
by  the  Fourier  transform  and  we  may  write  PWq  =  L 2  [—11,  ft]. 
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2.2  Reproducing  Kernel 

For  Hilbert  spaces  of  functions,  the  Riesz  representation  theorem  states  that  every  bounded 
linear  functional  may  be  represented  as  an  inner  product  with  a  particular  element  from  the 
Hilbert  space.  Let  ‘H  denote  the  Hilbert  space.  A  functional  on  7i  is  any  mapping  F  :  H.  i-*  (E . 
In  particular,  consider  the  collection  of  point  mappings  {Fx}x^g  given  as 

Vzes,  Fxf  =  f(x), 

where  Q  is  some  domain  on  which  functions  in  H  are  defined.  Clearly  the  point  mapping 
collection  {Fx}x^g  is  a  set  of  functionals  on  H.  It  is  easy  to  verify  that  each  member  of  this 
set  is  also  linear.  If  all  members  of  the  Hilbert  space  H  satisfy  a  pointwise  bound  on  Q 

V*  €  G,  \f(x)\  <  Mx\\f\\  (2.2.1) 

then  the  collection  [Fx}x^g  is  a  set  of  bounded  linear  functionals.  By  the  Riesz  representation 
theorem  for  each  x  £  Q  there  is  an  element  Kx  £  Ft  such  that  the  functional  Fx  may  be 
expressed  as 

Fxf  —  (/,  Kx) . 

The  reproducing  kernel  for  a  Hilbert  space  H(G)  is  a  mapping  1C  :  G  X  G  i-v  (D  defined  as 

fc(xiy)  —  {KyiKx)ft(gy 

If  the  pointwise  bound  of  (2.2.1)  holds  over  G  then  Ky(x)  =  A l(x,y)  for  x,y  €  G  and  H{G)  is  a 
reproducing  kernel  Hilbert  space  (RKHS). 

Example  2.2.1  For  O  >  0  the  Paley- Wiener  space  PWq  is  a  RKHS  with  reproducing  kernel 
)C(s,t )  =  d27rfi(t  —  s)  since  for  all  /  £  PWq 

fi*)  =  (fl[-u,U])  (t) 

=  (f  *  d2lTu)it) 

= 


where  Kt  =  Ttd27rn- 


Example  2.2.2  For  >  fix  >  0  the  Paley-Wiener  space  PWux,u2  is  a  RKHS  with  reproduc¬ 
ing  kernel  IC(s,t )  =  2cos(7r(0!  +  ft2)(i-  s))dT(Q2_0l)(t  -  s). 


2.3  Group  Representation 
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2.3.1  Groups 

A  group  is  a  set  Q  along  with  an  identity  element  and  an  associative  binary  operation  on  Q 
called  the  group  action.  The  group  action  is  denoted  as  the  operation  so  that  with  x,y  £  Q 
then  x  •  y  £  Q  denotes  x  acting  on  y.  The  identity  element  of  the  group  is  denoted  as  “e”,  and 
the  inverse  of  an  element  x  £  Q  is  a;-1.  To  formally  state  the  definition,  a  group  is  a  pair  (Q,  •) 
where  Q  is  a  set,  is  the  group  action  and  there  is  an  e  £  Q  so  that 

a.  if  x  £  Q  then  x  •  e  =  x,  and 

b.  if  x  £  Q  then  there  is  a  x~l  £  Q  so  that  x  •  x_1  =  e. 

A  group  is  abelian  if  the  group  action  is  commutative.  For  example,  (IR, +)  is  an  abelian  group 
with  identity  element  e  =  0.  Non-abelian  groups  will  be  of  use  in  the  general  approach  to 
discrete  signal  representation  presented  here.  Much  of  the  set  up  which  is  presented  is  adopted 
from  [HW89], 

The  two  examples  of  groups  which  follow  are  are  found  in  the  background  of  both  the 
Gabor  and  wavelet  theory.  The  particular  form  of  the  associated  group  operations  is  justified 
in  Appendix  B,  cf.  Section  2.3.3. 


Example  2.3.1  (Affine  Group)  The  affine  group  is  the  upper  half-plane  Qa  =  1RxIR+  along 
with  the  group  action  defined  as  follows.  Let  x,y  £  Qa  and  x  —  ( tx,sx )  and  y  =  (ty,sy)  then 

*  *  V  ~  (Gn  ~  {lx  “t~  Sx  ty ,  S-pSy') 


and 


X  —  —  (  Sxtx,Sx  ) 


so  that  the  identity  element  is  e  =  (0, 1). 


Example  2.3.2  (Weyl-Heisenberg  Group)  The  Weyl-Heisenberg  Group  is  Qjj  ~  T  X  IR  X 
]R  with  the  following  group  action.  Let  x,y  £  Qh  so  with  zx,zy  £  T  and  tx,jx,ty,jy  £  IR  we 
have  x  =  (zx,tx, /yx)  and  y  =  (zy,ty,  jy)  then 

x  •  y  —  {zx.,  tx, 'y j,)  •  {zy , tyi'yy)  —  {zxzye  x , “yx  *yy, tx  4-  ty) 


and 

{Zxitxilx)-1  =  {zxle~2vnxtx,-tx,-  7X) 
so  that  the  identity  element  is  e  =  (1,0,0). 


2.3.2  Weighted  Spaces 

Define  the  Hilbert  space  of  /x-square  integrable  functions  as 

Ll(G)  =  :  ]g\F{x)\2dn{x)  <  ooj 
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Let  F,G  E  L2{G).  The  space  Z/2(t7)  has  the  associated  norm 


11*1  Ilsw  = 


(Jg 


1 

2 


and  the  inner  product 


(F,G)Ll(g)  =  J  F(x)G(x)dp(x). 


A  measure  p  on  a  group  t/  is  left-invariant  if  for  every  integrable  function  F  on  Q  and  every 

y^Q 

J  F(y~1x)dp(x)  =  J  F(x)dp(x). 


Such  a  measure  is  called  a  left  Haar  measure  and  it  is  well  known  from  measure  theory  that 
such  a  measure  exists  and  is  unique  up  to  multiplication  by  a  constant. 

The  convolution  F  *  G  of  F  and  G  elements  of  L2(f7)  is  defined  as 


F*G(x)~  [  F{y)G{y  1x)dp{y). 
JQ 

For  x,y  £  G,  the  translation  operator  Tx  :  L^(Q)  L2((/)  is 


Txfiv)  =  f(x  lv )• 

If  fj,  is  the  left  Haar  measure  then  Tx  is  unitary  and  T~x  =  T*  =  Tx- 1.  The  involution  F  of 

F  e  LftG)  is 

F(x)  =  F(x_1). 

A  function  F  is  involutive  if  F  =  F.  The  convolution  F  *  G  may  also  be  written 

(F*G)(x)=(F,TxG}LW). 

In  the  case  that  Q  —  (1R,  +)  these  formulas -reduce  to  the  standard  ones  and  involutive  functions 
are  those  whose  real  parts  are  even  and  imaginary  parts  are  odd. 


2.3.3  Representation 

Let  7i  be  a  Hilbert  space  and  Q  be  a  group.  A  representation  of  Q  on  H  is  a  mapping 
n  :  Q  B(7i ,  H)  which  satisfies  the  relation 

n(x)n(j/)  =  n  (x-y), 

for  all  x,y  G  Q ■  For  example,  with  respect  to  the  group  (IR,  +),  n(t)  =  rt  is  a  representation  of 
1R  on  L2(1R)  since  rtrs  =  rt+s  where  <,s6l. 

An  element  g  £  H  is  cyclic  if  span  {n (x)g}x&g  =  H.  For  example,  G  PWq  is  cyclic 
with  respect  to  the  group  (IR,  +)  with  representation  tx. 

I.  A  representation  II  is  irreducible  if  every  g  E  'H\  {0}  is  cyclic. 
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II.  An  element  g  EfC  is  admissible  if 


/  \(g,H(x)9)\2dn(x)  <  oo. 

JQ 

The  set  of  admissible  functions  is  denoted  so  that 

AnAH)  =  {$€W\{0}  :  J^\(g,U(x)g)\2dp(x)  <  oo  j  . 

Note  that  An^ifH)  C  H. 

III.  A  representation  II  is  square-integrable  if  it  is  irreducible  and  An^iH.)  is  not  empty. 


2.4  Group  Representation  Transform 


The  basic  approach  to  discrete  representation  in  Hilbert  spaces  which  is  taken  here  involves 
the  sampling  of  an  auxiliary  signal  dependent  function  or  transform.  This  transform  is  known 
as  the  group  representation  transform.  It  is  a  generalized  transform  which  contains  the  wavelet 
and  Gabor  transforms  as  special  cases.  Feichtinger  and  Grochenig  [FG89]  have  developed  this 
generalized  approach  in  the  setting  of  Banach  spaces. 

The  group  representation  transform  [FG89]  of  /  £  i2(lR)  with  respect  to  an  admissible 
g  G  An^Oi)  is  a  mapping  Vg  :  I2(1R)  L\{Q)  given  as 

(V,/)(*)=(/,n(*)«>.  (2.4.1) 

The  group  representation  transform  is  the  fundamental  transformation  from  which  all  dis¬ 
cretizations  studied  in  this  thesis  will  come.  Discretizations  which  we  will  consider  are  of  the 
form  {(F!//)(a:)}xer  where  F  is  a  countable  subset  of  Q.  It  is  expected  that  such  samplings  of 
the  group  representation  transform  will  allow  full  recovery  of  /  under  certain  density  conditions 
on  T.  This  is  because  the  range  Vg(H)  is  a  reproducing  kernel  Hilbert  space.  To  see  this,  take 
f,g  EH  and  note  that  by  Cauchy-Shwarz 

I ( V)(*)l  =  I  </,  n(x)ff)  I  <  ll/ll  ||n(x)<7||  <  ll/ll  ||,||  ||H(*)||  <  Mx, 

where  Mx  <  oo  since  n(a;)  is  a  bounded  operator  by  definition.  Thus,  there  is  a  pointwise 
bound  of  the  form  given  in  (2.2.1).  From  the  discussion  in  Section  2.2  there  is  a  reproducing 
kernel  associated  with  Vg  and  the  range  Vg{H)  of  Vg  is  a  RKHS.  Moreover  with  the  aid  of  the 
following  theorem  it  is  possible  to  write  down  explicitly  the  reproducing  kernel  for  Vg(H). 

Theorem  2.4.1  ([GMP85])  Let  n  be  a  representation  of  the  group  Q  on  the  Hilbert  space 
%  with  left  Haar  measure  p.  If  n  is  square  integrable  then  there  exists  a  unique  self-adjoint 
positive  operator  T  :  >->•  H  such  that  for  all  <?i,  <72  G  An^W)  and  for  all  f\,  fa  €  H 

(Vgifi,Vg2f2)Ll{g)  =  (h,h)  {T9l,Tg2) . 
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As  consequences  of  this  theorem  two  corollaries  are  derived  immediately.  These  corollaries 
may  be  thought  of  as  counterparts  of  the  Parseval  and  Plancherel  relations.  With  T  be  the 
unique  self-adjoint  positive  operator  of  Theorem  2.4.1  the  positive  scalar  cg  is  defined  as  cg  = 

\\Tg\\. 

Corollary  2.4.2  (“Parseval”)  If  g  £  An,n(H)  and  fi,f2  G  H  then 

(VgfuVgh)Ll{Q)  =  c2g  <A,/2>. 

Corollary  2.4.3  (“Plancherel”)  If  g  6  An,n(%)  and  /  €  'H  then 

\\vgf\\ll(s)  =  4  ii/ii2. 

The  following  theorem  establishes  that  the  range  of  the  group  representation  transform  is 
a  reproducing  kernel  and  gives  explicitly  the  form  of  the  kernel.  Asserted  as  a  corollary  is  the 
fact  that  convolution  with  this  kernel  performs  the  orthogonal  projection  onto  the  range  of  the 
group  representation  transform. 

Theorem  2.4.4  Suppose  the  hypothesis  of  Theorem  2.4.1  are  satisfied  and  II  is  a  unitary 
representation  then  Vg(7i)  is  a  RKHS  with  kernel  JC(x,y)  =  TyK(x )  where 

K(x )  =  ^(Vgg)(x). 

cg 

Proof: 

Let  F  €  Ll(G). 

(F*K)(x)  =  [  F(y)K(y~1x)dn(y) 

JQ 

=  Jg  F(v )  (^(vg9)(y~lx^j  Mv) 

=  JgF(y)(g,n(y~lx)9)My) 

=  4/ F(y)  {TL(y)9^Yl(x)y}  My) 

C9  JG 

=  4  [  F(y)vg(n(x)yW(y) 

Cg  JQ 

9 

Orthogonally  decomposing  F  with  respect  to  Vg(7i)  as  F  =  Vgf  +  H  where  H  6  Vg{7i )-*-  for 
some  /  G  H  yields 

(F*K)(x)  =  1  (Vaf,Vg(H(x)g))L2{g)  (H  is  nullified) 

=  (/,  Il(a;)<7)  (by  Theorem  2.4.1) 

=  Vgf(x). 
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Corollary  2.4.5  Convolution  with  the  reproducing  kernel  K  performs  the  orthogonal  projec¬ 
tion  onto  the  range  Vg(H),  i.e., 

VFeLliG),  F  *  K  =  PVg(n)F. 


Note  that  the  kernel  K  is  involutive,  i.e.  K  —  K.  To  see  this  we  shall  without  loss  of 
generality  assume  cg  =  1.  Write 


K{x)  =  (Vag  r\x) 


v7a{x~l) 

(H(x)g,g) 
{g,H(x)g) 
(Vg9)(x)  =  K(x). 


Thus,  (F*K)(x)={F,TxK). 

2.4.1  Gabor  Transform 

With  g  £  L2(SV),  the  continuous  Gabor  Transform  Ggf  of  a  signal  /  £  T2(1R)  is 

(Ggf){t,7)  =  J  f(x)e~2m'yxg{x  -  t)dx. 

The  Gabor  transform  may  be  interpreted  as  time- varying  Fourier  transform  where  the  function 
g  acts  a  sliding  window  in  time  over  /.  At  a  particular  time  instant  t0  the  Gabor  transform  of 
a  function  /  is  the  Fourier  transform  of  /  modulated  by  a  t0  translated  version  of  g,  i.e., 

(Ggf){t0, 7)  =  ( fTtogy . 

In  this  way  the  Gabor  transform  attempts  to  expose  the  time-frequency  content  of  the  under¬ 
lying  signal  /.  This  transform  is  also  called  the  “short-time  Fourier  transform”. 

Alternatively  the  Gabor  transform  may  be  written  as 

(Ggf)(t,  7)  =  {f,e^Ttg) 

=  (/,n(M,7)p) 

=  (/>  n(ar)fif) 

where  x  =  (l,f,7)  £  Gh,  the  Weyl- Heisenberg  group,  and  n(z,t,7)  =  zr^g  where  \z\  —  1 
is  a  representation  of  Gh  on  T2(1R)  (See  Appendix  B  for  verification).  This  representation  is 
denoted  n  =  n# .  The  Weyl- Heisenberg  group  action  is  given  in  Example  2.3.2.  For  the  Gabor 
transform  the  associated  left  Haar  measure  is  the  product  measure 

d[i(z,t,  7)  =  dzdtd 7. 


19 


Although  the  toral  component  z  associated  with  the  Weyl-Heisenberg  group  Gh  is  necessary  to 
properly  define  the  representation  II#  it  can  effectively  be  ignored.  To  see  this,  first  note  that 
II#(2,t,7)  =  2lI#(M,7).  Hence,  for  any  g  E  i2(]R) 


/  II (x)gdfi(x)  =  I  dzdtdj 

JQ  J  T  J  R  J  R 

mzdzUH(l,t,j)gdtdj 

r 

=  [  [  TlH(l,t,i)gdtdj. 

J  R  J  R 

Theorem  2.4.6  ([HW89,  Proposition  3.2.4],[DGM86])  If  f,g  E  T2(1R)  then 


ll^/||ij(fc)  =  ||if||  ll/ll. 


This  theorem  may  be  used  to  show  that  H#  is  a  square  integrable  representation  of  Gh  on 
T2(]R).  Thus,  Theorem  2.4.1  is  applicable  where  T  =  Th  =  I-  All  g  E  T2(1R)  are  admissible 
and  Gg  is  a  multiple  of  an  isometry. 

2.4.2  Wavelet  Transform 

For  s  >  0  the  continuous  wavelet  transform  Wgf  of  a  signal  /  £  L2(Ul)  is 

(Wgf)(t,s)  =  si  J  f(x)g(s(x  -  t))dx. 

Alternatively,  the  wavelet  transform  may  be  written  as 

Wgf(t,s)  =  (/,  TtDsg) 

=  < f,E(t,s)g ) 

=  (f,U(x)g) 

where  x  =  ( t,s )  E  Ga >  the  affine  group,  and  n(t,s)  =  n>i(t,s)  =  rtDs  is  a  representation  of 
Ga  =  Iff  X  1R+  on  L2(1R).  The  affine  group  action  is  given  in  Example  2.3.1.  For  the  wavelet 
transform  the  associated  left  Haar  measure  is 

dfi(t,s)  =  s~2dtds. 

See  Appendix  B  for  verification  that  this  g  is  the  left  Haar  measure  and  that  II ^  is  a  represen¬ 
tation  of  Ga  on  T2(1R). 

Theorem  2.4.7  ([HW89,  Theorem  3.3.5],[GM84])  If  f,g  E  T2(  1R)  then 
\\Wgfhl{gA)  =  11/111,3  \\TAg\\2Hl  +  11/111,2  \\TAg\\2Hl, 
where  TAg  =  (l7l_2  sCt))  • 
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This  theorem  may  be  used  to  show  that  II ^4  is  a  square  integrable  representation  of  Ga  on 
H+  and  Hi.  Thus,  Theorem  2.4.1  is  applicable.  A  similar  result  can  be  developed  for  real 
signals,  c.f.  Remark  2.1.1.  By  Theorem  2.4.7  if  /  is  a  real  signal  then 

W,f\\LVs*)  =  ll/ll  II^A 

If  g  is  admissible  then  cg  =  ||TUff||  <  00  and  Wg  is  a  multiple  of  an  isometry. 


2.5  Sampling  and  Interpolation 

Let  H  be  an  arbitrary  Hilbert  space,  e.g.  PWq.  Let  Hd  be  an  arbitrary  discrete  Hilbert 
space,  e.g.  t2  (Z).  A  sampling  operator  is  any  mapping  L  :H  »-*■  Hd-  An  interpolating  operator 
for  H  is  a  mapping  A  :  Hd  >->  H . 

2.5.1  Sequences  in  Hilbert  space 

Definition  2.5.1  (Bessel  Sequence)  a.  {4>n}  C  H  is  a  Bessel  sequence  for  H  if 

v/ew,  £  !(/,&»>  I2  <  00. 

b.  {</>n}  CTtisa  Bessel  sequence  of  uniqueness  if  in  addition  to  a.  there  is  a  constant  A  >  0 
such  that 

A|i/ii2  <D</,«  i2- 


Definition  2.5.2  (Riesz-Fischer  sequence)  {0„}  C  H  is  a  Riesz-Fischer  sequence  if 


Vc  6  l2  (Z),  3  f£H  3  c  =  {(fM}. 


With  respect  to  the  sequence  {< fin }  C  H  define  the  sampling  operator  Lj,  :H  C2  (Z)  as 

L+f  =  {(fM} 

where  f  £  H.  The  notions  of  Bessel  and  Riesz-Fischer  sequences  may  be  directly  related  to 
properties  of  the  mapping  L#.  Namely,  {4>n}  is  a  Bessel  mapping  if  and  only  if  is  injective 
and  it  is  a  Riesz-Fischer  map  if  and  only  if  L $  is  surjective. 
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2.5.2  Density  of  Sampling  Sets 


Let  r  =  {tn}  C  Ht  be  a  real  sampling  set. 

I.  The  sequence  {tn}  is  uniformly  discrete  if  there  is  a  d  >  0  so  that  \tn  -  tm\  >  d  for  all 
n  ^  m. 

II.  A  uniformly  discrete  sequence  T  =  {t„}  is  uniformly  dense  with  density  Ar  if  there  is  an 
L  <  oo  so  that  for  all  n 

\tn  -  ra(AT)_1|  <  L. 

Additional  studies  of  sampling  in  Hilbert  spaces  can  be  found  in  [Beu61],  [Yen56],  [Beu66], 
[Yao67],  [Jer77],  and  [CA87], 
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Chapter  3 

Frames 


In  this  chapter  the  theory  of  (global)  Hilbert  space  frames  is  reviewed  and  some  necessary 
tools  are  developed.  The  theory  of  frames  is  due  to  Duffin  and  Schaeffer  [DS52],  cf.,  [Dau92], 
[DGM86],  [HW89],  [You80].  Throughout  the  chapter  H  denotes  a  Hilbert  space  contained  in 

T2(f/),  with  norm  ||. .  .||  =  ||. .  -Wl^q)  induced  from  L2^{Q).  Here,  Q  is  a  group  and  p  is  the  left 
Haar  measure. 


3.1  Frame  Basics 


As  a  concept,  frames  provide  an  intermediate  ground  between  the  two  related  notions  of 
completeness  in  a  space  and  an  orthonormal  basis  for  a  space.  Recall,  a  set  of  functions  {<j)n}  is 
complete  in  a  Hilbert  space  H  if  the  closure  of  their  span  is  the  whole  space  H,  i.e.  span  {(j>n}  = 
H.  A  set  {<j)n}  is  a  Schauder  basis  for  the  space  H  if  for  any  /  G  H  there  is  a  unique  set  of 
coefficients  {cn}  such  that  /  =  J2cn<f>n-  As  it  shall  be  seen  the  statements  that  a  set  {(j>n}  is 

a.  complete  in  7t, 

b.  a  frame  for  H, 

c.  a  Riesz  basis  for  H ,  and 

d.  an  orthonormal  basis  for  “H 

are  progressively  stronger.  In  other  words  d  =>•  c  =>  b  =>  a. 

To  any  set  {<f>n}  one  may  associate  the  operator  S  =  S<f,  defined  as 

sf  = 

The  frame  property  can  be  equivalently  characterized  in  terms  of  this  operator  S,  and  conse¬ 
quently  has  been  called  the  frame  operator.  Specifically  the  set  { <j>n }  is  a  frame  for  Ti  if  and 
only  if  there  are  constants  A,  B 

V/G  W,  A||/||2  <(/,  Sf)  <B\\f\\2 

such  that  0  <  A  <  B  <  oo.  Note  that  the  frame  operator  has  many  “nice”  properties  including 
linearity,  continuity,  and  invertibility.  Definition  3.1.1  gives  the  formal  definition  of  a  Hilbert 
space  frame,  its  associated  frame  operator,  and  the  notions  of  tightness  and  exactness. 
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Definition  3.1.1  a.  A  sequence  {<t>n}  C  Tt  is  a  frame  for  ft  if  there  exist  frame  bounds  A,B>  0 
such  that 

V/  6  ft,  A||/||2<^|(/,«|2<B||/f,  (3.1.1) 

where  summation  is  over  7L. 

b.  The  frame  operator  of  the  frame  {4>n}  is  the  function  S  :  ft  ft  defined  as  S f  — 

Yl  (fi  f'n)  (frn- 

c.  A  frame  is  said  to  be  a  tight  if  A  =  B. 

d.  A  frame  for  ft  is  said  to  be  exact  if  the  set  determined  by  the  removal  of  any  one  of  its 
elements  fails  to  be  a  frame  for  ft. 


Some  examples  of  frames  are  presented  below.  From  these  examples  it  is  seen  that  tightness 
and  exactness  are  independent  properties. 

Tight  and  Exact  Frame  Clearly  any  orthonormal  basis  {en}  C  ft  is  a  tight  exact  frame 
with  bounds  A  =  B  =  1  since  by  Parseval’s  equality  for  orthonormal  bases  1  (/>  4>n)  |2  =  ||/||2- 
It  is  exact  since  the  elements  are  orthogonal  and  the  removal  of  any  one  element  will  cause  the 
reduced  set  to  fail  to  be  dense  in  ft. 


Tight  and  Non-exact  Frame  The  union  of  any  finite  number  N  >  2  of  orthonormal  bases 
in  7 i  yields  a  tight  non-exact  frame  for  ft  with  frame  bounds  A  —  B  =  N .  To  see  this,  for  each 
m  =  1,2,...,  A  let  {emjn}n  be  an  orthonormal  basis  for  TL.  Then  by  Parseval’s  equality  for 
orthonormal  bases 

E  ll/il2  =  wii/ii2. 

771  =  1  n  771  =  1 

Since  each  basis  {ern,n}n  is  dense  in  H  the  removal  of  any  one  element  from 

N 

l_J  iem,n} 

771  =  1 

will  result  in  a  collection  of  vectors  which  contains  at  least  one  orthonormal  basis  for  H .  There¬ 
fore  it  is  dense  in  H. 

For  example  {ci,Ci,C2,e2, C3,e3,.-.}  is  a  tight  non-exact  frame  with  A  =  B  =  2. 


Non-tight  and  Exact  Frame  A  non-tight  and  exact  frame  may  be  generated  from  { <t>n }  as 


where  {an}  is  a  sequence  of  scalars  satisfying  0<A<a£<f?<oo  and  there  is  a  pair  m,n 
such  that  am  j -  an.  Clearly  then  the  set  {anen}  is  exact  and 


X)l  if,anen)  |2  = 

n 

< 

< 


e»)i2 

n 

Y.  (S“I> "t)  l(/.e»)|2 
BY,  l(/,«»)l2  =  -BH/ll2- 

n 
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The  lower  bound  is  similar. 

For  example  {2ei ,  e2, e3,  €4, . . is  a  non-tight  exact  frame  with  A  —  l  and  5  =  2. 

Non-tight  and  Non-Exact  Frame  There  are  many  ways  to  generate  non-tight  non-exact 
frames.  For  example,  the  union  of  a  orthonormal  basis  with  a  scaled  orthonormal  basis.  An 
example  is 

{ei,2ei,e2,2e2,e3,2e3,. . .}  . 

The  following  theorem  exhibits  the  close  relationship  between  exact  frames  and  orthonormal 
bases.  More  precisely  it  states  that  exact  frames  are  Riesz  bases.  Riesz  bases  are  by  definition 
related  to  orthonormal  bases  by  a  topological  isomorphism.  See  [You80,  p.  188]  for  a  proof  of 
the  equivalence  of  parts  a  and  b. 

Theorem  3.1.2  Let  H  be  an  arbitrary  Hilbert  space  and  {<j>n}  be  a  sequence  of  elements  in 
' H .  The  following  are  equivalent: 

a.  {4>n}  is  an  exact  frame  for  H. 

b.  {< t>n }  is  a  Riesz  basis  for  H. 

c.  {<f> n}  is  a  bounded  unconditional  basis  for  H. 


The  following  theorem  states  some  of  the  fundamental  properties  of  frames. 

Theorem  3.1.3  a.  If  {<A„}  C  H  is  a  frame  with  frame  bounds  A,  5,  then  S  is  a  topological 
isomorphism  with  inverse  5-1,  is  a  frame  with  frame  bounds  5_1  and  A-1,  and 

v/ew,  /  =  £(/,  s-'<j>n)  <t>n  =  E  (/,  <M  5- v„  (3.1.2) 

in  7 i. 

b.  If  {(j>n)  C  H,  let  L  :  H  t2  (2)  be  defined  as  Lf  =  {(/,  cf .,  (3.2.3).  If  {4>n}  is  a 
frame  then  S  =  L*L,  where  L*  is  the  adjoint  of  L. 


Since  the  frame  operator  S  may  be  factored  [DGM86,  Dau90]  as  L*L  an  immediate  conse¬ 
quence  is  that 

(f,Sf)  =  (f,L*Lf)  =  (Lf,Lf)  =  \\Lf\\2. 

Since  { 4>n }  is  a  frame  with  frame  bounds  A  and  B  this  implies  that 


Thus, 


A||/||2  <  IIT/II2  <  5||/||2. 

||T||  <  and  \\L~X\\  <  A~^ , 
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where  L~l  is  defined  on  the  range  L(H). 

It  is  clear  that  if  A  and  B  are  frame  bounds  for  a  frame  {4>n)  then  any  other  pair  A\  and 
Bi  such  that  0  <  A\  <  A  and  oo  >  B\  >  B  are  also  valid  frame  bounds  for  {<f>n}-  It  is  of 
interest  to  know  the  smallest  upper  bound  and  the  largest  lower  bound  which  serve  as  frame 
bounds  for  a  frame.  This  motivates  the  notion  of  the  best  frame  bounds.  Given  a  frame  {</>„} 
for  a  Hilbert  space  H  with  frame  operator  S,  the  best  bounds  A  and  B  are 


A  =  inf 

fen 

B  =  sup 

fen 


(Mf) 
ll/ll2  ’ 
(/.  Sf) 


Since  ||A/||2  =  (/,  Sf)  it  follows  that  the  best  bounds  A, B  are  also  A  =  \\L  x||  2  and  B  =  ||£||2. 


3.2  Frame  Representation 


In  this  section  the  natural  discretization  suggested  by  the  frame  operator  is  exposed  via 
Theorem  3.1.3b.  Consider  the  operator  L  of  Theorem  3.1.3  and  its  adjoint.  The  theorem 
asserts  that  L  and  its  adjoint  L*  are  factors  of  the  frame  operator  5.  Explicitly,  the  operators 
L:H^£2  (Z),  and  L*  :  £2  (Z)  H  are 


(3.2.1) 


and 


L*c  =  J2c^n,  (3.2.2) 

where  f  £fi  and  c  £  £2  (Z).  It  is  easily  verified  that  in  fact  S  —  L*L  since  for  all  /  6  H 

Sf  =  '£(f’<t>n)<t>n  =  L*Lf- 


The  desired  discretization  operator  A  is  a  mapping  from  H  to  i2  (Z)  and  is  defined  as 

L 


U 

/► 


l2(  Z) 


(3.2.3) 


Figure  3.1  depicts  the  mappings  L  and  its  adjoint  L*.  If  { 4>n }  is  a  frame  for  H  then  the  mapping 
defined  in  (3.2.3)  is  called  the  frame  representation  or  frame  discretization  operator.  The  frame 
representation  operator  L  plays  a  central  role  in  Theorem  3.1.3.  Part  a  of  the  theorem  describes 
one  method  to  recover  a  signal  f  £  Ti  from  its  frame  representation  Lf  £  l2  (Z).  In  part  b, 
the  theorem  indicates  that  the  frame  operator  S  has  factors  L  and  L*.  In  addition,  Theorem 
3.2.1  below  states  that  the  frame  representation  operator  L  has  an  inverse  when  considered  on 
the  range  L(H).  These  facts  form  the  basis  for  the  iterative  reconstruction  scheme  given  in 
Proposition  3.4.3  and,  in  turn,  the  notion  of  the  frame  correlation  operator  discussed  in  Section 
3.3. 


A  characterization  of  frame  representation  operators  is  given  in  the  following  theorem. 
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Figure  3.1:  The  mappings  L  and  L* 


Theorem  3.2.1  ([Ben93,  Theorem  3.6])  The  sequence  {<f>n}  is  a  frame  for  H  if  and  only  if 
the  mapping  L  given  in  (3.2.3)  is  a  well  defined  topological  isomorphism  onto  a  closed  subspace 
of  i2  (Z). 


Theorem  3.2.1  has  three  significant  consequences  worthy  of  mention.  Namely,  if  £  is  a  frame 
representation  operator  then 

a.  L  is  injective  (one  to  one), 

b.  L{H)  is  closed,  and 

c.  L*  is  surjective  (onto). 

To  see  the  injectivity  of  the  map  L  suppose  {<f>n}  is  a  frame  for  H  with  bounds  A  and  B  and 
that  Lf\  =  Lfi-  Then 

0  =  \\Lh  -  Lf2\\2  =  ||Z,(/i  -  f2) ||2  >  A\\fx  -  h\\2 

which  implies  that  /i  =  /2.  Thus,  L  is  injective.  Because  L  is  an  injective  bounded  linear 
operator  whose  range  L(H),  is  closed  then  L*  is  onto  by  Proposition  A.l. 

3.3  Frame  Correlation 

A  concept  which  arises  naturally  in  frame  theory  is  the  notion  of  frame  correlation  given  in 
Definition  3.3.1.  It  will  be  seen  that  the  frame  correlation  is  a  crucial  element  in  the  process  of 
reconstruction  from  frame  representations,  cf.  Proposition  3.4.2. 

Definition  3.3.1  (Frame  Correlation)  Let  be  a  frame  for  the  Hilbert  space  %  with 
frame  representation  operator  L.  The  frame  correlation  operator  is  defined  as  R  =  LL*. 
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The  frame  operator  S  —  L*L  and  the  frame  correlation  R  =  LL*  are  similar  objects  and 
play  similar  roles  in  the  theory  of  frames.  In  fact,  a  reconstruction  theory  may  be  developed 
without  ever  introducing  the  frame  correlation,  cf.  [FG92].  Hence,  one  may  ask  why  the  frame 
correlation  is  an  important  object  to  study?  To  answer  this  note  that 

S  :H^H, 


while 

R  :  L(H)  ~  L(H). 

In  many  cases  of  interest  H  will  be  an  infinite  dimensional  Hilbert  space  having  elements  which 
can  not  be  directly  processed  by  a  digital  machine  while  L(H)  will  consist  of  discrete  elements, 
i.e.  countable  sets,  which  (if  truncated)  may  be  processed  digitally.  For  example,  such  an  H 
is  the  space  of  bandlimited  functions  PWq,  and  L{H)  =  { f(tn )  :  /  £  PWn}  for  some  sequence 
{tn}.  Thus  the  operator  S  does  not  admit  a  digital  implementation  while  R  does. 

3.3.1  Properties 

The  frame  correlation  matrix  R  shares  many  properties  of  the  frame  operator  S.  For 
instance,  they  are  both  non-negative  self  adjoint  operators  which  map  bijectively  onto  their 
range.  The  crucial  differences,  however,  are  that 

a.  the  range  and  domain  of  R  is  contained  in  t2  (2),  and 

b.  the  range  of  R  need  not  be  all  of  t2  (Z). 

These  two  differences  directly  relate  to  the  issues  of 

a.  digital  implementability  and 

b.  representation  noise  robustness 

respectively.  As  discussed  previously  the  fact  that  R  operates  on  countable  sequences,  i.e. 
digital  signals,  immediately  suggests  that  it  is  possible  to  implement  R  on  a  digital  machine. 
This  is  not  possible  for  S.  Robustness  to  noise  in  the  representation  is  directly  related  to  the 
size  of  the  kernel  of  R.  This  issue  is  addressed  in  Section  3.5.1. 

This  section  is  composed  of  several  propositions  which  illustrate  the  properties  of  a  frame 
correlation  R.  The  first,  Proposition  3.3.2  exposes  the  matrix  representation  of  the  frame 
correlation  R  (of  a  frame  {<f>n }  for  a  Hilbert  space  7i)  as  the  Gram  matrix  associated  with  the 
sequence  of  elements  {</>„}  C  7i.  The  second,  Proposition  3.3.3  compiles  a  list  of  useful  general 
properties  of  a  frame  correlation  R.  Thirdly,  Theorem  3.3.4  establishes  the  ramifications  for  R 
and  other  equivalences  if  the  underlying  frame  is  exact. 

Proposition  3.3.2  Given  a  frame  { <f)n }  for  the  Hilbert  space  H,  the  frame  correlation  matrix 
R  has  the  matrix  representation 


R  —  ((^mj^n))  —  Rm,n • 


Proof: 
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Take  any  c  G  t2  (Z).  Then  one  can  write 


Rc  =  LL*c  =  {(L*c,<}>n)}  = 


{firm  *^n) 


2  /  cm  {firm  fin)  i 

m 


so  that 


and 


(• Rc)n  =  (££*c> 


{</>  Oi 

o)  (<i>o,<f>o) 

A-iA  1)  A0A1) 


\ 

(  :  \ 

A  \A-\) 

c- 1 

{4>  iA  o)  ••• 

Co 

{<f>  iA  i) 

Cl 

V  :  / 

(3.3.1) 


(3.3.2) 


■ 


Proposition  3.3.3  Suppose  {</>n}  is  a  frame  for  the  Hilbert  space  TL  with  frame  representation 
operator  L ,  correlation  R  and  bounds  A  and  B. 

a.  If  H  is  infinite  dimensional  then  L  is  not  compact;  and  thus,  R  is  not  compact. 

b.  For  each  row  m  of  R ,  limn_+00  — ►  0. 

For  each  column  n  of  R ,  limm_>00  \Rm,n\  -*  0. 

c.  If  the  set  {</>„}  is  an  orthonormal  basis  for  H  then  the  frame  correlation  operator  is  the 
identity. 

d.  ker  R  =  L{U)l. 

e.  R  maps  L(7i)  bijectively  to  itself. 

f ■  R  =  Pl(h)R  =  RPL(H) 

g.  R  is  self  adjoint. 

h.  R  >  0. 

Proof: 

a.  To  prove  the  non-compactness  of  L  a  normalized  sequence  {/„}  C  L2(]R)  shall,  be  chosen 
such  that  {Lfn}  has  no  converging  subsequence.  Let  fn  —  l[nn+i]  so  that  ||/„||  =  1.  For  such 
a  sequence  one  has 

||  Lfn  -  Lfm  ||  =  ||L(/n  -  /m)||  >  AS\\fn  -  fm\\, 

where  the  equality  holds  by  the  linearity  of  L  and  the  inequality  holds  by  the  frame  condition 
applied  to  fn  —  fm.  For  our  choice  of  {/«}  it  is  easy  to  see  that  ||/n  —  /m||  =  2(1  —  ^m,n).  Thus 
one  has  that 

Vn^m,  \\Lfn- Lfm\\>2A%. 

Since  l2  (Z)  is  complete  one  may  conclude  that  {Lfn}  has  no  converging  subsequences.  Hence 
L  is  not  compact. 

To  show  R  is  not  compact  note  that  by  part  e.  there  is  a  well  defined  inverse  R~l  of  R  on 
L{7i).  With  {/„}  as  in  part  a.  define  cn  =  [|^-i^yn||  =  R~1L(fn/an )  where  the  normalization 
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is  well  defined  since  an  =  ||R  lLfn ||  =  ||i^,/n||  >  B  2  >  0.  Also  we  have  an  <  A  2  or 
a~ 2  >  A.  As  in  part  a.  we  may  compute 

||/n/®ra  /m/®m||  —  (®n  d"  )(1  $ m,n )  ~  2A(1  ^m,n)- 

For  the  sequence  {cn}  C  S?  (Z)  then 

Vn  7"  1 1 -RCtj,  i2cm||  —  ||-^'(/n/®n  /m/®m)||  ^  2A2, 

and  one  concludes  that  {Rcn}  has  no  converging  subsequences  so  that  R  is  not  compact. 

b.  Since  {fn}  is  a  frame  for  H  then  one  must  have  in  particular  that 

Vm  <  5||<?!)m||2. 

n 

For  the  sum  to  converge  it  is  necessary  that  |  (<f>m,<l>n)  |  —*  0  as  m  — >  00. 

c.  Follows  from  Proposition  3.3.2  since  (^>m,<^n)  =  t>m,n- 

d.  i/  ker  R  C  L(H)l 

Let  Co  G  ker  R  so  that  Rcq  =  0.  We  will  show  for  all  /  £  H  that  (c0,  Lf)  =  0.  Since  L*  is 
onto  then  for  all  /  £  H  there  is  a  c  £  L(H)  so  that  /  =  L*c.  Thus, 

(c0,  Lf)  =  (I’co,/)  =  (L*co,L*c)  =  (Rc0,c)  =  0. 

ii /  ker  12  D  L(H)X 

Let  c±  £  L(H)X .  For  any  c  £  S?  (Z)  we  have 

(■ Rc±,c )  =  ( cx,Rc ) 

=  (c±,L(L*c))  =  0. 

Since  (Rc±,c)  =  0  for  all  c  £  l2  (Z)  then  Hex  =  0. 

e.  Since  L  is  a  linear  injective  map,  one  need  only  demonstrate  RL  is  an  injective  map  from 
L(H)  to  L(7i)  to  prove  that  R  is  injective  on  L{7i).  Write  RL  =  LL*L  =  LS.  Since  both  L 
and  S  are  injective  then  LS  =  RL  is  also.  Thus  R  is  1-1.  Since  R  is  self  adjoint  and  1-1  then 
it  must  also  be  onto  by  Corollary  A. 2. 

f.  Let  c  £  l2  (Z).  Clearly,  Rc  =  LL*c  £  L(7i).  Thus,  R  —  P^^R  and  by  taking  the  adjoint 

R  =  RPl(h)- 

g.  R*  =  (LL*)*  =  LL*  -  R. 

h.  Since  L*  is  surjective  for  all  /  £  L2(M)  there  is  a  c  £  L(H)  such  that  /  =  L*c.  Thus, 
0  >  ||/||2  =  (L*c,  L*c)  =  (c,  LL*c)  =  (c,  Rc). 


We  have  seen  in  Proposition  3.3.3e  that  R  maps  L(H)  bijectively  to  itself.  If  L(H)  is  all  of 
£2  (Z)  ( L  is  onto)  then  R  is  topological  isomorphism  on  l2  (Z).  This  can  only  happen  if  the 
underlying  frame  is  a  Riesz  basis,  i.e.  an  exact  frame.  This  and  other  equivalences  are  the 
content  of  Theorem  3.3.4. 

Theorem  3.3.4  Let  {4>n}  be  a  frame  for  Tt  with  frame  representation  operator  L ,  frame  cor¬ 
relation  R ,  and  bounds  A  and  B.  The  following  are  equivalent: 
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a.  L  is  onto  i 2  (Z). 


b.  L*  is  one  to  one. 

c.  R  is  a  topological  isomorphism  on  £2  (Z). 

d.  R  >  0 

e.  {4>n}  is  a  Riesz-Fischer  sequence. 

f.  {(f>n)  is  a  Riesz  basis. 

Proof: 

(a.  =►  b.) 

From  elementary  operator  theory, 

ker  L*  =  L(H)l  =  {£2  (Z))x  =  {0} 

so  that  L*  (a  linear  operator)  is  one  to  one. 

(b.  =►  c.) 

Clearly  R  6  B(7i,'H)  since  R  =  LL*  and  L  e  B(H,H).  R  is  one  to  one  because  L  is  one  to 
one  (always)  and  by  assumption  L*  is  one  to  one  so  that  the  composition  LL*  is  also  one  to  one. 
R*  =  R  is  onto  by  Proposition  A.l  since  R(H)  —  L(H)  is  closed  (Theorem  3.2.1b).  Thus  R  is 
bijective  on  t2  (Z)  and  there  exists  an  inverse  J?-1  on  all  of  t2  (Z).  Moreover,  ||R_1||  <  A~l  by 
dual  frame  arguments.  Thus,  R  is  a  topological  isomorphism  on  i2  (Z). 

(c.  . y  a.) 

Suppose  R  is  a  topological  isomorphism  and  L  is  not  onto  i 2  (Z).  But  R(H)  =  L{H)  ^ 
f.2  (Z)  which  means  that  R  can  not  be  onto.  This  contradicts  the  assumption  that  R  is  a 
topological  isomorphism. 

(b.  =►  d.) 

Write 

(c,  Rc)  =  ( c,LL*c )  =  \\L*c\\2  >  0. 

Since  L*  is  one  to  one  then  (c,  Rc)  =  0  if  and  only  if  c  =  0.  Thus,  R  >  0. 

(d.  =»  b.) 

R  >  0  means  that  for  all  non-zero  c  6  l2  (Z) 

0<  (c,  Rc)  =  \\L*c\\2. 

If  L*c\  =  L*C2  then 

(ci  -  C2,R(C!  -  c2))  =  \\L*Ci  -  L*c2 II  =  0 
and  we  conclude  that  c2  —  c2.  Thus,  L*  is  one  to  one. 

(a.  e.) 

By  definition  {</>„}  is  a  Riesz-Fischer  sequence  means  that  for  all  c  6  f2(Z)  there  is  a 
f  £  “H  such  that  c  =  {(/,  </>«)}•  With  L  the  frame  representation  Lf  =  {(/,  <fin)}  this  translates 
equivalently  to  the  statement  that  L  is  onto  i2  (Z). 
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(c.  =►  f.) 

Suppose  R  is  a  topological  isomorphism  on  £ 2  (Z).  We  have  Lf  =  {(f,4>n)}.  Fix  an  integer 
N  and  define  a  new  operator  L jy  as  Ljqf  =  {{f,<f>n)}n^N-  We  show  that  the  removal  of 
one  element  from  the  set  {</>„}  will  fail  to  be  a  frame  by  constructing  a  non-zero  f  EH  so  that 
LnI  —  0.  Let  {en}  be  an  the  standard  orthonormal  basis  for  £2  (Z),  i.e.  en  —  {£TO)„}.  Since  R 
is  a  topological  isomorphism  on  £ 2  (Z)  then  R  has  an  inverse  R~x  on  l2  (Z).  We  may  then  pick 

/  =  L*R~xeN 


so  that  for  this  choice  of  / 

Lf=  L(L*R~xeN)  =  eN 

and  hence  L/v/  =  0. 

(f.  =>  c.) 

By  definition  {<f>n}  is  an  exact  frame  means  there  exists  a  topological  isomorphism  T  and  an 
orthonormal  basis  {en}  for  H  so  that  for  all  n,  <j)n  =  Ten.  Let  Le  be  the  frame  representation 
associated  with  the  orthonormal  basis  { en }.  Since  {en}  is  an  orthonormal  basis  then  Le  is  a 
topological  isomorphism  from  H  onto  l2  (Z).  Similarly  L*  is  a  topological  isomorphism  from 
£ 2  (Z)  onto  H.  Noting  that 

LeT*f  =  {(T7,e„)}  =  {(f,Ten)}  =  «/,^)}  =  Lf 

we  conclude  that  L  =  LeT*  and  R  =  LL*  =  LeT*TL*.  Thus,  since  each  factor  of  R  is  a 
topological  isomorphism  R  is  a  topological  isomorphism  on  l2  (Z). 


3.3.2  Pseudo-Inverse 

Proposition  3.3.3e  implies  that  R  has  an  inverse  on  L(H).  This  inverse  is  denoted  R~x  and 

VceL(7f)  c  =  R-lRc  =  RR~lc. 

To  extend  the  inverse  to  all  of  (?  (Z)  define  the  pseudo  inverse 

Rt  ^  R-xPL(n)  (3.3.3) 

where  P^H)  *s  the  orthogonal  projection  operator  onto  the  image  of  L.  Definition  3.3.5  gives 
a  definition  of  pseudo-inverse.  We  note  that  there  are  several  equivalent  definitions  with  inter¬ 
esting  interpretations  of  the  pseudo-inversion  process,  viz.  Appendix  C. 

Proposition  3.3.3f  can  be  used  to  demonstrate  that  as  defined  in  (3.3.3)  R *  is  in  fact  a 
bonified  pseudo-inverse.  Using  Proposition  3.3.3f  we  have  that  for  all  c  E  £2  (Z) 

R]Rc  =  R~xPl^Rc  =  R~xRPL(H)c  =  PL{n)C 


and  similarly 

RR^c  —  RR~X  Pl(-h)c  =  Pl(H)c • 
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We  conclude  that  R^R  =  RR^  =  Pl(H)  •  This  observation  makes  the  verification  of  the  defining 
conditions  of  a  pseudo  inverse  in  Definition  3.3.5  trivial.  Thus,  3.3.3  does  indeed  give  the  pseudo¬ 
inverse  of  R.  In  Section  3.4  an  iterative  method  for  the  construction  of  the  pseudo-inverse  is 
presented. 

Definition  3.3.5[Gro77,  Definition  (P)]  If  T  E  B(Ki,7t2)  has  closed  range,  then  T*  is  the 
unique  operator  in  satisfying 

(1)  TTt  is  self  adjoint, 

(2)  T^T  is  self  adjoint, 

(3)  TT*  T  =  T,  and 

(4)  T+TT+  =  T+. 


The  following  theorem  shows  that  the  best  frame  bounds  of  a  frame  are  directly  related  to 
the  operator  norms  of  the  frame  correlation  R  and  its  pseudo  inverse  R\ 

Theorem  3.3.6  Let  {</>„}  be  a  frame  for  a  Hilbert  Space  H  with  best  frame  bounds  A  and  B 
and  frame  correlation  R.  Then  the  frame  correlation  R  is  related  to  the  frame  bounds  A  and 
B  as 

a  -A  = 

b.  B  =  ||A||. 


Proof: 

a.  Since  {</>„}  is  a  frame  for  H  then  L*  is  surjective.  Hence,  for  all  /  £  H  there  is  a  c  6  L(H) 
so  that  /  =  L*c  and  one  may  write  for  /  not  zero  a.e. 


(f,Sf)  ( L*c,SL*c )  _  (L*c,L*LL*c)  _  (c,R2c) 
ll/ll2  {L*c,  L*c)  ( c,LL*c )  (c,Rc) 

for  c  off  the  kerT*  =  L(H)± ,  i.e.  c  6  L(H).  The  surjectivity  of  L*  then  implies  the  best  lower 
frame  bound  A  is 


inf 

fen 


ll/ll2 


inf 

c£L(H)  (c,  Rc) 


Since  R t  is  onto  L(H)  for  any  c  €  L(H)  there  is  a  Co  €  L(H)  so  that  c  =  R^cq.  With  this 
substitution  one  has 


(c,  i?2c)  (^R^co,  R2R^co^  (co,co) 

{ c,Rc )  (Rlc0,RRic0}  (Etc0,c0)' 

Thus, 

A=  inf  _  ( znv  (r'c°’c°)Y'  _  ( :av 

coeL(H)  {Ric0,c0)  yCoez4W)  (co,c0)  J  ycoeL(«)  (co,co)  J 
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where  the  last  equality  follows  since  R t  is  a  non-negative  operator.  Further, 


A  = 


sup 

i  Co  £L(W)©keri?t 


|(fltco,co)| 

(co,c0) 


-1 


(r)c0,cq) 

<*€*(Z)  (C0,C0) 


=  sup 


=  ii^ir1. 


Where  one  has  used  the  fact  that  ker  =  ker  R  =  L(H)l  from  Proposition  3.3.3d.  The  last 
equality  is  a  consequence  of  Fact  2.1.3. 


b.  The  upper  bound  can  be  proven  by  via  the  dual  frame  {5  l<t>n}  with  best  bounds  A',  B' 
and  frame  correlation  R' .  Since  R'  =  and  A'  =  B~x  application  of  part  a.  to  the  dual  frame 
yields  B~l  =  ||(J8/)t||“1  or  B  =  ||i?||. 


3.3.3  Duality 

From  Theorem  3.1.3a  a  frame  {<£„}  has  an  associated  dual  frame  {ipn},  where 
and  S  is  the  frame  operator.  As  a  frame,  {ipn}  also  has  a  frame  representation  operator  L^, 

where  L^f  =  {(/,  ipn}}  —  {(/,  6'_1</>n)} .  As  a  matter  of  notation  we  may  write  both  L  and  Lp 
to  indicate  the  frame  representation  with  respect  to  the  frame  {<£„}.  Since  S  is  a  topological 
isomorphism  clearly  L,p(7i)  =  L^(Ti).  With  this  notation,  Equation  3.1.2  may  be  written  as 

V/  6  H,  f  =  L%Upf  =  L^f.  (3.3.5) 

From  this  observation  we  may  conclude  that 

l;l^  =  =  / 

where  /  is  the  identity  operator  on  H.  Further,  if  attention  is  restricted  to  the  range  L(H)  we 
may  write 

T—  1  —  T* 

L<t>  ~  L<1> 

and 

=  LV 

Moreover,  the  relation  between  the  frame  representation  L p  and  its  dual  is 

Lip  —  R  i  RL^p  -  R  ^  L,p  =  R  * L,p , 

/ 

where  R  is  the  frame  correlation  associated  with  the  frame  {<f>n}.  Table  3.1  lists  the  relationships 
between  the  frame  bounds  A  and  B ,  frame  operator  S,  frame  representation  L,  and  frame 
correlation  R  of  a  frame  {<j>n}  and  its  dual  frame  {6’“Vn}-  Dual  quantities  are  denoted  with 
e.g.  S'  is  the  dual  frame  operator. 
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Elements 

Bounds 

Operator 

Representation 

Correlation 

WEEM 

(A,B) 

S 

L 

R 

■SUSHI 

(B~1,A~1) 

S -1 

RTXL 

Rt 

Table  3.1:  Relation  of  frame  objects  and  their  duals. 


3.4  Iterative  Reconstruction 


Let  H  be  a  Hilbert  space  and  take  /*  £  H  arbitrarily.  Assume  also  that  {<j>n}  is  a  frame  for 
H  with  frame  bounds  A  and  B,  frame  representation  X,  and  frame  correlation  R.  This  section 
details  an  iterative  procedure  for  the  recovery  of  a  signal  /*  from  its  frame  representation  L /». 
The  iterative  procedure  will  generate  a  sequence  { cn }  C  L(H)  which  converges  to  a  c*  (E  L(H) 
such  that  /*  =  X*c*.  Moreover  the  sequence  converges  at  an  exponential  rate.  The  algorithm 
for  the  computation  of  the  sequence  {cn}  may  be  implemented  digitally  and  identified  with  the 
computation  of  the  inverse  R -1  of  the  frame  correlation  R. 

3.4.1  Frame  Operator 

It  may  be  easily  shown  [Ben92,  Algorithm  50]  [Ben93,  Section  6.6]  that  since  {4>n}  is  a  frame 
for  H  with  frame  bounds  A,B 


II I- 


2 

A  +  B 


■S||< 


B  -  A 
A  +  B 


<  1, 


so  that  by  the  Neumann  expansion, 


A  +  B 


E(J- 


j=o 


A+B 


sy 


(3.4.1) 


where  I  is  the  identity  operator  in  7 i.  For  any  /*  G  7i  applying  (3.4.1)  to  5/*  yields 


/*  =  E(/-AW's)/*»  (3.4.2) 

3=0 

where  A  =  2/(A  +  B). 

An  iterative  procedure  for  the  recovery  of  /*  from  5/*  could  be  constructed  by  3.4.2  as  a 
difference  equation,  e.g.  [FG92],  [CFS].  However,  such  a  procedure  would  not  be  directly  digi¬ 
tally  implementable  in  the  case  that  the  underlying  Hilbert  space  H  consists  of  analog  signals. 
Instead  we  will  focus  on  iterative  methods  for  reconstruction  which  are  digitally  implementable. 
These  methods  utilize  not  the  frame  operator  S  but  the  frame  correlation  R. 

3.4.2  Frame  Correlation 


With  a  view  toward  digital  implementation  it  is  desirable  to  construct  an  iterative  algorithm 
for  the  recovery  of  /*  from  X/*.  To  do  this  we  first  shown  that  I  —  A R  is  a  contraction  on  L(H). 


Lemma  3.4.1  Let  {<£n}  be  a  frame  for  H  with  frame  representation  operator  X,  correlation 
R  and  bounds  A,  B.  If  0  <  A  <  2/1?  then  ||/—  <  1  and  ||X  —  A-RH^J)  =  1  if  { 4>n }  is 

not  exact.  In  particular  one  may  take  A  =  2/(A  +  B). 
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Proof: 

Let  V  and  R'  be  the  dual  frame  representation  and  correlation  respectively.  Since  ( V)*  is 
surjective  then  for  any  /  G  H  there  is  a  c  6  L\7i)  so  that  /  =  ( L')*c .  This  together  with  the 
fact  that  {5-1^n}  is  a  frame  for  H  yields 

B~l  ( c,R'c )  <  (^c,(R')2cj  <  A-1  ( c,R'c ).  (c.f.  Equation  (3.3.4)) 

Letting  c  =  {R')^Cq  one  has 

B~l  (Rc0,c0)  <  (c0,co)  <  A -1  (Rc0,c0) . 


For  all  nonzero  cq  G  L'(H)  this  means 


Thus  one  has  for  A  >  0 


1  -  A  B  < 


A  <  <  a, 

\C0) Co) 

((/-AJ2)co,co)  < 


(cO>co) 


and  combining  the  facts  that  I  —  A R  is  self  adjoint,  cf.,  Proposition  3.3.3g,  and  fact  2.1.3 

|| I  -  A#||£(W)  =  sup  ^)c>c)l  <  max{|l  -  AA|,|1  -  A£|}.  (3.4.3) 

ceL(H)  <c-c> 


We  would  like  to  find  a  lambda  such  that  J|7  -  A R\\l(h)  <  !•  This  condition  is  satisfied  for  all 
A  G  (0,2 / B).  In  particular  if  A  =  2/ {A  +  B)  then 

|1  -  XA\  =  |1  -  \B\  =  (B  -  A) /(A  +B)<  1. 


For  this  choice  of  A  we  have  proven  that  ||J  -  XR\\l(H)  <  1.  Clearly,  for  all  nonzero  c  taken  from 
ker  R  =  L( 77)-*-  one  has  =  1.  Since  t2  (Z)  =  L(H )  ©  L(H )x  then  ||/  -  A7?||^Z)  =  1 

if  7/(77)-*-  =  kerX*  ^  {0},  i.e.  by  Theorem  3.3.4  {4>n}  is  not  exact. 


Proposition  3.4.2  The  signal  /*  may  be  recovered  from  its  frame  representation  L /*  as 

OO 

/.  =  A  J2  L*(I  -  \R)jLf*,  (3.4.4) 

3=  0 

where  L*c  =  cn<f>n  for  c  =  {cn}. 

Proof:  Since  (Lf,c)  =  (f,L*c)  and 

(T/,c)  =  =  (/,5Zcn^„), 

one  obtains  the  formula  for  L*c. 
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Because  of  (3.4.2)  and  the  fact  that  S  =  L*L ,  it  is  sufficient  to  prove 

OO  OO 

\J2L*(I-\R)jLf*  =  V(J  -  XL* L)j(XL*L)f*. 

j=0  j=0 


The  j  =  0  terms  are  clearly  the  same  in  (3.4.5).  Assume 

XL* (I  -  X R)jLf*  =  (/  -  AX*X)J'(AX*X)/*. 


(3.4.5) 


(3.4.6) 


Then,  using  (3.4.6),  compute 


XL*(I  -  XR)j+1Lf*  = 


XL*{I  -  A R)iLU  -  XL*{I  -  XR)jXRLf * 

A (/  -  A L*L)jL*Lf*  -  A (/  -  XL* Ly  L* L{\L* Lf«) 
X (I  -  X L*L)j(I  -  X L*L)L*Lfm 
A(7-  XL*L)j+1L*Lf^ 


and  the  result  follows  by  induction. 


Proposition  3.4.2  leads  directly  to  Algorithm  3.4.3  which  details  an  iterative  reconstruction 
procedure  for  the  recovery  of  the  signal  /*  from  its  frame  representation  X/*.  Moreover,  this 
iterative  procedure  will  converge  at  an  exponential  rate. 


Algorithm  3.4.3  Let  {<j>n}  be  a  frame  for  a  Hilbert  space  H  with  frame  representation  L, 
correlation  R  and  bounds  A,  B.  Suppose  Co  =  X/#  is  the  frame  representation  of  a  signal 
/*  G  H.  set  /o  =  0.  If  A  =  2 /(A  +  B )  and  hn,  cn  and  fn  are  defined  recursively  as 

hn  =  A  L*cn, 

A  _  T, 

Cn+1  —  On  ijilni 

/n+l  —  fn  A  hn , 

then 

a.  lim  /„  =  /*,  and 

b.  <  f  «ni  where  a  =  ||7  -  AE||i(w)  <  1. 

Proof: 

a.  An  elementary  induction  argument  shows  that 


Vn, 


fn+1  =  AX*  -  X Ry 


\j=o 


Co- 


Consequently,  by  Proposition  3.4.2,  one  has 


lim  fn  =  /*. 
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b.  Write 


II  fn  ~  /*|| 


—  ||(/n+l  “  fn)  +  (/n+2  ~  /n+l)  +  (fn+ 3  ~  fn+ 2)  •  •  -|| 

<  EllA+i-AII 

k>n 

=  \\\L*(I  -  \R)nLf*\\ 

k>n 

fc>n 

<  \B  (  £  «" )  ll/.ll 

\k>n  ) 

=  (l^)  ab|I/>11 

<  ^«n||/*||  (by  Equation  (3.4.3).) 


Algorithm  3.4.3  underscores  the  importance  of  the  correlation  frame  operator  R  in  the 
reconstruction  process.  Formally  one  may  rewrite  (3.4.4)  as 

/*  =  L*R~lLf*.  (3.4.7) 

Note  that  if  R  is  known  apriori  the  inverse  frame  correlation  R~r  can  be  computed  once  (olf-line) 
and  stored  for  future  reconstruction  computations  via  (3.4.7). 

A  crucial  element  in  determining  convergence  of  the  algorithm  presented  in  Algorithm  3.4.3 
was  the  fact  that  co  =  A/*  and  in  particular  that  Co  G  L{'H).  In  fact,  if  c0  is  not  entirely  in 
Z/(7f )  then  the  algorithm  will  not  converge,  cf.,  Lemma  3.4.1.  This  issue  is  addressed  in  Section 
3.5  where  a  second  algorithm  is  presented  which  will  converge  on  all  of  l2  (Z). 

Remark  3.4.4  It  is  important  to  note  that  Algorithm  3.4.3  may  be  implemented  digitally. 
That  is,  the  sequence  {]C£=1  ck}  may  be  generated  entirely  on  a  digital  processor  from  the 
initial  representation  Cq.  In  practice  the  iterative  algorithm  will  be  terminated  at  some  finite 
iteration  N.  The  sequence  cfc}  is  related  to  /n  as  /jv  =  L*  Y^k-ick-  The  final  step 

in  converting  the  digital  representation  Y^k= l  ck  to  In  involves  a  generalized  digital  to  analog 
procedure  well  determined  by  L*. 


3.5  Noise  Robustness 


With  respect  to  frame  representations  of  signals  there  are  two  domains  in  which  noise  may 
perturb  a  signal.  The  first  space  is  the  Hilbert  space  H  C  L2(  1R)  and  the  second  is  its  image 
under  £,  i.e.,  L(Tt )  S=  £2(Z).  The  former  space  is  called  the  signal  space  and  the  latter  the 
coefficient  space. 
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3.5.1  Coefficient  Noise  and  Non-exactness 

Suppose  f  £  H  is  a  signal  of  interest  with  frame  representation  Lf  £  L{H).  There  are 
various  sources  of  perturbation  which  may  occur  in  the  coefficient  domain.  For  instance,  the 
representation  Lf  may  be  communicated,  stored,  or  retrieved  incorrectly.  A  more  direct  issue 
is  the  representation  of  the  real  or  complex  valued  coefficients  Lf  on  a  finite  precision  digital 
machine.  Such  a  representation  requires  quantization  of  the  coefficient  sequence.  Letting  Q  : 
IR  i — >  {/x,  l2, . . . ,  12n}  C  IR  be  quantization  by  a  fixed  number  N  of  bits,  we  have  at  our  disposal 
not  Lf  but  Q(Lf).  Quantization  may  be  viewed  as  noise  in  the  coefficient  domain. 

It  is  natural  to  investigate  the  robustness  of  frame  representations  with  respect  to  coeffi¬ 
cient  noise.  Clearly,  frame  representations  would  be  of  little  use  if  small  perturbations  in  the 
coefficient  domain  destroyed  large  amounts  of  information  in  the  signal  domain.  The  following 
argument,  cf.,  [DGM86]  and  [Dau92,  Section  3.6],  asserts  that  frames  which  are  far  from  ex¬ 
act,  i.e.  have  a  large  degree  of  linear  dependence,  exhibit  robustness  to  noise  in  the  coefficient 
domain. 

Suppose  one  has  a  noise  corrupted  version  c  of  a  signal  /,  so  that 

c  =  Lf  +  d, 

where  d  is  random  noise  which  may  be  uniformly  distributed.  Reconstructing  from  c  entails 

L*B)c  =  L*R'1PL(n)(Lf  +  <*„  +  dx)  =  L*R-\Lf  +  dy), 

where  d  =  dy  +  dx  is  the  orthogonal  decomposition  of  d  with  respect  to  L ,  i.e.  dy  6  L(H)  and 
dj_  £  L(7i)±.  Thus,  all  the  noise  energy  outside  the  range  L(H)  is  automatically  nullified  in  the 
reconstruction  process.  For  a  uniformly  distributed  perturbation  d,  the  larger  the  kernel  of  L* 
the  more  energy  in  the  noise  will  be  nullified  and  the  more  noise  tolerance  will  be  achieved.  Since 
the  kernel  of  L*  being  large  is  equivalent  to  the  underlying  frame  having  a  large  degree  of  linear 
dependence  one  may  conclude  that  non-exactness  yields  robustness  to  coefficient  imprecision. 

Tacitly  assumed  in  the  above  argument  is  a  method  for  the  reconstruction  of  a  signal  from 
its  corrupted  frame  representation.  The  reconstructed  version  /j  from  a  noise  perturbed  frame 
representation  c  —  Lf  -\-  d  is 

/t  =  L*  R)c. 

Note  that  c  need  not  be  in  the  range  L{H).  In  Section  3.4  an  iterative  algorithm,  Algorithm 
3.4.3,  was  presented  for  the  reconstruction  of  a  signal  from  its  uncorrupted  frame  representa¬ 
tion.  However,  as  was  shown  earlier,  this  algorithm  will  not  converge  for  arbitrary  initial  data, 
i.e.,  representations  outside  the  range  L(H).  Here,  a  second  algorithm  is  provided  which  will 
converge  on  all  of  l2  (Z).  This  algorithm  is  given  as  Algorithm  3.5.3. 

Our  approach  is  to  modify  Algorithm  3.4.3  so  as  to  alleviate  the  problem  of  initialization 
with  a  coefficient  sequence  outside  the  range  of  L.  As  a  first  step  note  that  Rc  £  L(H)  for 
all  c  £  l2  (Z).  Thus,  if  c0  were  initialized  to  Rc  then  all  that  would  be  required  is  to  provide 
an  algorithm  for  the  computation  of  R~2  on  L(H)  instead  of  IE-1  on  L(H).  Before  such  an 
algorithm  may  be  presented,  it  is  necessary  first  to  attend  to  some  technical  details. 

Lemma  3.5.1  Let  Hi  and  H2  be  two  Hilbert  spaces.  If  A  6  B(Hi,H2)  then 

\\I-A*A\\<1  «=}>  Vx  £  H\,  0  <  inf  ||Az||  <  sup  ||j4z||  =  ||A||  <  >/2, 

IMM  ||a:||=i 
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where  /  is  the  identity  operator  on  H\. 

Proof: 

Since  I  —  A* A  is  self  adjoint,  by  Fact  2.1.3 

\\I-A*A\\  =  sup  \(x,(I  -  A*A)x)\ 
ll*ll=i 

=  sup  li-ll'MI2!- 
twi=i 

Therefore  the  condition  that  ||/  —  A*A\\  <  1  is  equivalent  to  the  two  conditions 

sup  (1  -  ||Aa:||2)  =  1  -  inf  \\Ax\\2  <  1 
||a;||=l  ll*l|— 1 

and 

sup  (||Aa;||2  -  1)  =  sup  ||v4x||2  -  1  <  1, 
ll*ll=i  ll*ll=i 

which  further  translate  to 

inf  ||v4x||2  >  0  and  sup  ||Aa;||2  <  2. 

11*11=1  ||a;||=l 


■ 


With  the  aid  of  Lemma  3.5.1  it  can  easily  be  shown  that  the  operator  R~ 2  may  be  approx¬ 
imated  as  a  Neumann  series.  To  do  this  it  is  first  shown  that  for  a  proper  choice  of  relaxation 
parameter,  e.g.,  A  =  V2 /(A  +  B)  the  operator  I  -  (A R)2  is  a  contraction  on  the  range  L{H). 
This  is  the  content  of  Lemma  3.5.2. 


Lemma  3.5.2  Let  {</>„}  be  a  frame  for  H  with  frame  representation  operator  L,  correlation  R 
and  bounds  A,  B.  If  A  =  y/2/(A  -(-  B)  then  ||7  —  (All)2|k(H)  <  1. 

Proof: 

Take  A  =  A R  and  Hi  —  Hi  =  L{H)  in  Lemma  3.5.1.  Then 


PII  =  AP||  = 


V2 

A  +  B 


B  <V 2 


and 


inf 

ceL(W),||c||=l 


||Af2c||  >  0 


since  by  Proposition  3.3.3e  R  is  1-1  on  L(H)  and  ||c||  =  1.  The  result  follows  from  application 
of  Lemma  3.5.1. 


Let  A  =  \Z2/(A  +  B).  Since  I  —  (AT?)2  is  a  contraction,  the  Neumann  series 

OO 

a2£(/-(A-R)2)* 

k= 0 
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will  converge  to  R  2  on  L('H).  Formally,  with  c  an  arbitrary  element  from  l2  (Z)  and  cq  =  Rc 

A2^(/-(AjR)Yc0, 

k= 0 

will  converge  to  ( R~2)Rc  since. Co  £  L(H).  In  fact,  it  shall  be  seen  that 

Rf  =  flim)A2^(/-(AJR)2)fcf?  (3.5.1) 

n->°°  k= 0 

where  R 1  is  the  pseudo  or  generalized  inverse  [Gro77]  of  R.  It  is  easy  to  check  that  as  defined 
in  Equation  3.5.1  R t  does  indeed  satisfy  the  requirements  of  Definition  3.3.5. 

Algorithm  3.5.3  Let  {4>n}  be  a  frame  for  a  Hilbert  space  H  with  frame  representation  L, 
correlation  R,  and  bounds  A  and  B.  Suppose  c  £  t2  (Z)  is  the  corrupted  frame  representation 
of  a  signal  /*  £  H.  Set  Co  =  Rc  and  /o  =  0.  If  A  =  y/2/(A  +  B)  and  hn,  cn  and  fn  are  defined 
recursively  as 

K  =  A  2L*cn, 

^n+i  =  RLhr D 

/n+1  =  fn  +  hn, 

then 

a.  lim  fn  —  f^  =  L*R^c  and 

b.  \\fj  —  fn\\  <  Man,  where  M  <  oo  and  a  =  ||7  -  (Ai2)2||  <  1. 

Proof: 

a.  As  in  Algorithm  3.4.3,  an  elementary  induction  argument  shows  that 
Vn,  fn+ 1  =  A 2r  ^£(7  ~  (Ai2)2)J  J  c0. 

Consequently  one  has 

lim  fn  =  L*R*c. 


b.  Write 

ll/n+1  -  /nil  =  ||M  =  ||Ai*(7  -  (Ai7)2)”c||  <  A2||X*||  (|| J  -  (Ai2)2||)”  \\Rc\\  <  M'an , 
where  M'  =  A253/2||c]|  <  oo  since  c  £  l2  (Z).  Thus, 

||/t  -  /n||  <  £  =  M'TZ-  = 

fc>n  1 


Iterative  processes  for  the  construction  of  generalized  inverses  have  been  well  investigated, 
e.g.,  [Sho67],  [Pet67],  [Alt60],  cf.,  [Gro77]  for  a  broad  overview. 
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3.5.2  Signal  Noise  and  Frame  Coherence 


Intuitively,  the  noise  in  a  signal  is  that  part  which  lacks  structure  or  coherence.  For  example, 
in  auditory  signals  one  can  readily  identify  noise  with  incoherent  garbled  or  hissing  sounds. 
Implied  in  the  use  of  the  term  “coherence”  is  some  point  of  reference  with  which  other  points 
are  coherent.  It  seems  reasonable  to  adopt  a  (properly  chosen)  set  of  primitive  functions  as 
such  a  reference  point.  In  speech,  examples  of  coherent  structures  are  the  phonemic  primitives 
which  have  been  learned  by  the  individual;  and  a  pertinent  collection  of  primitive  functions 
may  be  extracted  from  the  study  of  the  cochlear  mechanics  of  ear  [BT92a,  BT92b]. 

Suppose  { 4>n }  is  a  collection  of  appropriate  primitive  functions.  In  simple  terms,  a  signal 
/  is  coherent  with  respect  to  the  set  {</>„}  if  that  signal  may  be  approximated  well  by  a  linear 
combination  of  a  relatively  small  number  of  members  of  { <pn }.  Mallat  [MZ92a]  has  used  a 
similar  notion  with  respect  to  arbitrary  “dictionaries”  of  functions.  Clearly  such  a  primitive 
set  can  not  be  chosen  arbitrarily  if  it  is  required  that  /  may  be  approximated  by  a  linear 
combination  of  its  members.  In  addition,  for  practical  implementations  it  is  required  that  such 
approximations  may  be  carried  out  in  a  numerically  stable  manner.  Both  of  these  requirements 
are  satisfied  if  the  reference  set  {< j>n }  is  a  frame  for  a  large  enough  Hilbert  space,  i.e.  one  which 
contains  all  signals  of  interest. 

For  these  reasons  attention  shall  be  limited  to  collections  {<j>n}  which  are  frames.  Conse¬ 
quently,  considered  as  “non-noisy”  are  those  signals  which  are  coherent  with  respect  to  the 
frame  {<t>n}-  This  view  of  coherent  signal  versus  noise  admits  a  relatively  simple  procedure  for 
the  recovery  of  a  signal  embedded  in  noise.  This  procedure  is  now  outlined. 

Let  Ti  C  L2(IR.)  be  a  Hilbert  space  of  interest  and  {</>„}  C  H  be  a  frame  for  H  with  frame 
representation  operator  L.  Suppose  one  has  a  signal  /*  £  H  which  is  coherent  with  respect 
to  { 4>n }•  Let  f w  be  a  noise  corrupted  version  of  /*,  where  fw  =  /*  +  w  and  w  £  L2(1R)  is 
non-coherent  with  respect  to  Note  that  fw  is  not  necessarily  contained  in  7 i. 

From  the  discussions  in  Section  3.4  and  Section  3.5.1,  if  {</>„}  were  a  frame  for  all  of  L2(1R) 
then 


fw  =  L*B)Lfu 


would  perfectly  recover  the  noisy  signal.  Since  our  main  objective  is  to  reject  the  noisy  portion 
w ,  this  approach  is  clearly  inappropriate  without  modification.  The  question  naturally  arises 
as  to  whether  there  is  some  processing  which  may  be  performed  in  the  coefficient  domain  L(H) 
which  will  act  as  a  noise  suppressant  in  the  signal  domain  H.  This,  in  fact,  is  the  crux  of  our 
approach. 

Consider  a  truncation  operator  F  :  L(7i)  FL{H)  which  nullifies  or  truncates  coefficient 
sequences  in  places  where  the  representation  L /*  of  /*  has  small  coefficients  (less  than  a  small 
positive  real  number  6).  That  is  for  any  c  £  L( H) 


f  cn,  |(L/*)„|  >  6 
1  0,  otherwise 


Clearly,  the  operator  F  =  Ffmt$  depends  on  the  signal  /*  and  the  threshold  6.  It  is  easy  to 
establish  that  F  is  a  linear  bounded  operator.  In  fact  ||Fj|  =  1.  In  Chapter  5  properties  of  such 
truncation  operators  are  fully  developed. 

Now  consider  a  reconstruction  procedure  which  starts  not  from  the  whole  sequence  Lfw  but 
from  the  truncated  sequence  FfWi$Lfw.  In  this  case  we  may  define  a  reconstruction  f$  of  the 
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coherent  signal  /*  from  knowledge  of  its  noise  corrupted  version  fw  as 

U  =  L*R*FfwtSLfw. 

Define  the  6  truncated  version  of  the  sequence  Lfw  as  eg  =  F/W7gLfw.  Expanding  fw  and 
using  the  linearity  of  the  operators  FfWts  and  X  we  have 

cs  =  Ffw,&L(f*  +  w)  =  FfWtsLf «  +  FfWtsLw. 

The  hope  of  this  scheme  for  noise  suppression  lies  in  the  expectations  that 

a.  for  a  coherent  signal  /*  the  truncation  FfWtgLf *  «  X/*,  and 

b.  for  a  non-coherent  signal  w  the  truncation  FfWtgLw  «  0. 

Consequently  we  expect  that  eg  ~  X/,.  In  this  case  we  may  view  eg  as  a  noisy  version  of  X/ 
and  use  Algorithm  3.5.3  with  initial  data  Co  =  eg  to  perform  the  noise  suppression. 

Let  us  look  more  deeply  into  the  question  of  why  this  thresholding  procedure  should  be 
expected  to  perform  noise  suppression.  If  {4>n}  is  a  frame  for  TL  with  frame  representation 
operator  X  then  there  is  the  following  norm  equivalence  property.  Namely,  there  are  positive 
scalars  A  and  B  such  that 

V/  €  H,  A\\f\\2  <  ||X/||2  <  B||/||2. 

We  may  interpret  such  a  relationship  as  an  approximate  energy  preservation  between  the  do¬ 
mains  H  and  Thus,  for  all  signals  /  G  ,  the  representation  Lf  must  have  approximately 

the  same  energy  as  the  original  signal;  however,  the  distribution  of  the  energy  among  the  co¬ 
efficients  in  Lf  is  dependent  on  the  signal’s  degree  of  coherence  with  the  underlying  frame 
{</>„}.  In  fact,  for  a  signal  /*  which  is  coherent  with  respect  to  the  frame  {</>„},  i.e.,  one  which 
is  represented  by  a  linear  combination  of  a  relatively  few  members,  the  frame  representation 
norm  equivalence  necessarily  implies  that  these  few  coefficients  must  contain  most  of  the  signal 
energy  and  hence  have  a  relatively  large  magnitude.  Similarly,  a  pure  noise  signal  w,  being 
incoherent  with  respect  to  the  set  {< },  must  have  a  frame  representation  in  which  the  noise 
energy  is  spread  out  over  a  very  large  numer  of  coefficients.  Hence,  these  coefficients  must  have 
a  relatively  small  magnitude.  In  light  of  this,  if  we  now  consider  the  noise  corrupted  signal 
fw  =  f*  +  »,  for  an  appropriate  value  of  the  threshold  6  the  small  coefficients  in  Lfw  due  to 
the  non-coherent  portion  w  will  be  suppressed  while  the  larger  coefficients  due  to  the  coherent 
portion  fw  will  be  preserved  under  truncation.  Thus,  the  iterative  Algorithm  3.5.3  initialized 
with  Co  =  FfWigLfw  is  in  fact  a  technique  for  the  suppression  of  noise. 

Note  that  these  arguments  depend  strongly  on  the  notion  of  coherence,  and  more  specificly 
coherence  with  respect  to  a  particular  frame.  It  should  be  clear  that  given  a  particular  ap¬ 
plication  appropriate  frames  must  be  chosen.  For  example,  in  the  context  of  music  processing 
there  is  a  natural  coherence  with  respect  to  the  notion  of  time  frequency  evolution.  A  natural 
description  of  a  piece  of  music  involves  the  idea  of  frequencies  moving  in  time.  If  our  objec¬ 
tive  is  to  remove  noise  from  music,  or  audible  signals  in  general,  then  it  seems  reasonable  to 
ask  for  frames  which  are  well  localized  in  both  time  and  frequency.  This  naturally  leads  to 
incorporation  of  wavelet  or  Gabor  frames  as  the  reference  for  coherence. 

Truncations  of  this  form  are  one  of  the  motivating  factors  spawning  the  notion  of  a  “local” 
frame.  Roughly  speaking  a  local  frame  for  a  Hilbert  space  H  is  a  frame  for  a  much  smaller 
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space  H(f *)  in  H  generated  by  a  specific  signal  /*  6  H.  It  is  also  required  that  /»  be  contained 
(almost)  in  7f(/*).  Local  frames  are  the  subject  of  Chapter  5.  We  note  that  numerical  examples 
of  the  performance  of  the  noise  suppression  abilities  of  such  a  scheme  are  presented  in  Section 


6.3.1. 


3.6  Numerical  Verification 


In  this  chapter  two  algorithms  have  been  presented  for  the  recovery  of  a  signal  from  its  frame 
representation.  It  has  been  established  that  Algorithm  3.4.3  will  converge  to  the  original  signal 
provided  that  it  is  initialized  with  a  valid  frame  representation,  i.e.  a  coefficient  sequence  entirely 
in  the  range  of  the  frame  representation  operator  L.  If,  however,  the  frame  representation  is 
perturbed  so  that  it  is  no  longer  in  the  range  of  L  then  this  algorithm  will  not  converge. 
This  situation  spawned  the  development  of  a  second  algorithm  which  would  contend  with 
perturbations  in  the  frames  representation  which  fall  outside  the  range  of  the  operator  L. 
It  has  been  established  that  Algorithm  3.5.3  will  always  converge  to  a  best  approximation  to 
the  signal  given  its  perturbed  frame  representation. 

To  illustrate  the  situation  we  perform  the  following  experiment.  We  generate  two  frames  4>x 
and  $2  such  that  4>i  has  no  redundancy  and  that  4>2  has  a  bit  of  redundancy.  More  precisely 

for  its  span  {$1}  C  ]R20  and  a  redundant 


{< i>- 


m 

'n  > 


20 


/  n— 1 


we  construct  a  non-redundant  frame  4>i 

frame  4>2  =  for  its  span  {$2}  C  ]R20  such  that 

•  both  and  4>2  have  the  same  frame  bounds  ( A,B )  =  (0.5, 1.5),  and 


•  dunker#!  =  20  and  dimker#2  =  19. 

where  R\  and  #2  are  the  frame  correlations  of  4>i  and  <b2  respectively.  Thus,  there  is  no 
redundancy  associated  with  the  frame  $i  while  there  is  one  degree  of  redundancy  associated 
with  the  frame  4>2. 

Let  L\  and  L2  be  the  respective  frame  representations  associated  with  4>i  and  $2.  Next  we 
choose  a  Co  6  1R20  as 

c0  =  (111...1). 

Thus  Co  is  contained  in  Li(H)  but  Co  is  not  contained  in  L2(7f). 

The  results  of  initializing  both  Algorithm  3.4.3  and  Algorithm  3.5.3  with  c0  is  displayed  in 
the  figures.  In  Figure  3.2  the  top  graph  displays  the  norm  of  the  sequence  {c„}  as  a  function 
of  the  iteration  number  when  Algorithm  3.4.3  is  initialized  with  c0  for  the  frame  $x.  The 
bottom  graph  displays  the  norm  of  the  sequence  {cn}  as  a  function  of  the  iteration  number 
when  Algorithm  3.5.3  is  initialized  with  Co  for  the  frame  4>x. 

In  Figure  3.3  the  top  graph  displays  the  norm  of  the  sequence  (cn)  as  a  function  of  the 
iteration  number  when  Algorithm  3.4.3  is  initialized  with  Co  for  the  frame  4>2.  The  bottom 
graph  displays  the  norm  of  the  sequence  (cn)  as  a  function  of  the  iteration  number  when 
Algorithm  3.5.3  is  initialized  with  Co  for  the  frame  4>2. 

Recall  that  in  each  algorithm  a  sequence  {/„}  is  generated  which  is  given  by 

fn  =  L*jr  Cn. 
k= 0 
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Figure  3.2:  Comparison  of  the  {||cn||}  for  the  case  of  the  non-redundant  frame  §1. 


Thus  for  convergence  we  must  have  cn  -»  0.  In  the  figures  we  have  plotted  ||cn||  as  a  function 
of  the  iteration  number.  Since  the  vertical  axis  on  the  graph  is  logarithmic  a  negatively  sloped 


line  indicates  an  exponential  rate  of  decay. 

For  the  frame  $1  we  can  see  from  Figure  3.2  that  both  algorithms  exhibit  the  predicted 
rate  of  exponential  decay  for  {|M>.  Both  algorithms  are  expected  to  converge  because  c0  * 
in  the  range  of  Lx.  For  the  frame  $2,  however,  c0  is  not  in  the  range  of  l2  and  as  a  result  i  is 
expected  that  Algorithm  3.4.3  will  not  converge.  These  expectations  are  confirmed  in  igure 
3.3.  Figure  3.3  clearly  shows  the  Mowing: 


Figure  3.3:  Comparison  of  the  {|jcn||}  for  the  case  of  the  redundant  frame  *2. 
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a.  Algorithm  3.5.3  convergence  properties  are  identical  to  non  redundant  case,  while 

b.  Algorithm  3.4.3  fails  to  converge  since  ||cn||  approaches  a  constant. 

Thus,  the  experiment  confirms  the  predictions  of  the  theory. 
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Chapter  4 

Sampling  in  RKHS 


With  the  material  of  the  previous  chapter  as  motivation,  a  generic  discrete  represenation 
of  analog  signals  is  developed  and  relations  are  made  to  frames  and  irregular  sampling  in 
certain  reproducing  kernel  Hilbert  spaces.  The  discrete  representations  studied  here  come  from 
samples  of  group  representation  transforms  on  lattices  contained  in  appropriate  groups.  These 
representations  are  quite  general  and  rely  on  a  familiarity  with  the  material  in  Section  2.3.  In 
the  last  sections  of  this  chapter  three  special  cases  of  discrete  representations  are  studied  in 
detail.  Respectively,  these  three  special  cases  result  in  the  study  of  Paley- Wiener  frames,  Gabor 
frames,  and  wavelet  frames. 


4.1  The  Generic  Representation 


Let  Ji  be  a  Hilbert  space  contained  in  T2(1R)  and  let  Q  be  a  group  with  representation  n 
on  H.  Suppose  g  is  an  admissible  function,  i.e.  g  6  Au,^(Td)  where  p  is  the  left  Haar  measure. 
The  generic  discrete  representation  of  a  signal  /  £  H  (in  general)  is  an  irregular  sampling 
over  the  group  Q  of  its  group  representation  transform  Vgf  £  LffQ).  Recall  that  the  group 
representation  transform  Vg  is  a  mapping  Vg  :  H  *->■  L2(t/)  given  by  (2.4.1)  redisplayed  here  for 
convenience 

C Vgf)(x)  =  (f,n(x)g ). 


For  a  generic  discrete  representation  the  sampling  set  T  is  some  countable  irregular  set  of 
points  from  the  group  Q 

r  =  {xn}  c  g.  (4.1.1) 

Thus,  T  is  a  discrete  lattice  contained  in  the  group  Q .  The  corresponding  lattice-  value  pair 
A (/)  is 

A(f)  =  (T,{Vgf(x)}xer).  (4.1.2) 

A (/)  is  a  lattice  in  Q  together  with  the  group  representation  transform  of  /  evaluated  at  all 
the  points  on  that  lattice.  The  fundamental  concern  of  discretization  theory  is  the  following 
question: 

What  conditions  on  the  sampling  set  T  allow  the  recovery  of  f  from  A (/)? 
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As  we  shall  see  this  question  is  related  to  the  frame  properties  of  the  collection 

{nCaOffW  •  (4.1.3) 

4.1.1  Frames  and  Sampling  in  Vg(W) 

In  Chapter  3  it  was  seen  that  collections  of  elements  from  a  Hilbert  space  which  are  frames 
have  associated  discrete  representations  characterized  by  a  topological  isomorphism  L  on  a 
closed  subspace  of  i2  (Z).  As  a  result  there  are  numerically  stable  iterative  algorithms  for  the 
reconstruction  of  signals  starting  from  their  frame  representations. 

An  underlying  theme  in  sampling  theory  is  the  connection  between  boundedness,  reproduc¬ 
ing  kernels,  and  sampling  expansions.  In  Section  2.2  it  is  shown  that,  via  the  Riesz  representa¬ 
tion  theorem,  the  condition  that  all  members  of  a  functional  Hilbert  space  satisfy  a  pointwise 
bound  implies  a  reproducing  kernel  for  that  space.  Thus,  loosely  speaking  we  may  say  that 
boundedness  leads  to  reproducing  kernels.  In  Section  2.4  such  a  pointwise  bound  is  easily 
established  (via  Cauchy-Shwarz)  for  members  of  Vg(H).  The  reproducing  kernel  for  Vg(H)  is 
given  explicitly  in  2.4.4  as 

K  =  cg(Vgg), 

where  cg  is  some  constant.  The  accompanying  reproducing  formula  is 

VF  €  Vg(H),  F(x)  =  ( F  *  K){x)  =  (F,  TgK)^g) 

for  all  x  G  Q.  From  this  we  may  conclude  that  knowledge  of  the  samples  {  F(xn)}  of  a  function 
F  G  Vg('H.)  is  equivalent  to  knowledge  of  the  inner  products  j(F,  TXn K)li 

In  the  spirit  of  Chapter  3  this  observation  leads  us  to  consider  discretizations  C  :  Vg(7i)  •-> 
i2  (Z)  defined  as 

CF  =  {(F,TXnK)Ll{g)}  =  {F(xn)} 
with  the  adjoint  C*  :  t2  (Z)  >->■  Vg(TL) 

C*c  =  '£cnTXnK, 

where  F  G  Vg(7{)  and  c  =  {cn}  G  i2  (Z).  Recall  that  for  a  fixed  xq  G  Q  the  xo-translation  TXoF 
of  a  function  F  G  L2(Q)  is  TXoF(x)  =  F(xq1x).  Because  CF  consists  of  irregular  samples  of  F, 
this  discretization  allows  us  to  develop  expansions  which  recover  F  from  its  irregular  samples 
in  Q  provided  the  collection 

{T*„K} 

is  a  frame  for  a  large  enough  subspace  of  Vg(7i).  For  example,  if  {TXnK}  were  a  frame  for 
Vg(H )  then  any  F  G  Vg(H)  could  be  reconstructed,  viz.  (3.4.7),  from  CF  as 

F  =  C*K\CF), 

where  1Z  =  CC*  is  the  frame  correlation.  Since  CF  —  {F(xn)}  this  is  a  sampling  expansion  for 
reconstructing  a  function  F  which  is  an  element  of  the  reproducing  kernel  Hilbert  space  Vg{'H) 
from  its  sample  values.  Together  with  the  previous  boundedness  arguments,  this  observation 
justifies  the  statement  that  boundedness  implies  RKHS  implies  sampling  expansions. 
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Our  main  interest  is,  however,  not  to  reconstruct  group  representation  transforms  from  their 
irregular  samples.  Our  main  interest  is,  rather,  to  develop  discrete  representations  of  signals 
from  H  with  associated  reconstruction  algorithms.  It  turns  out  that  irregular  samples  of  group 
representation  transforms  are  in  fact  exactly  the  type  of  discrete  representation  which  we  seek. 
The  two  are  essentially  related  by  the  group  representation  transform  and  its  “inverse”.  To 
make  this  notion  of  inverse  precise  Proposition  4.1.1  identifies  the  proper  spaces  in  which  the 
group  representation  transform  has  an  inverse. 

Proposition  4.1.1  The  group  representation  transform  Vg  has  a  well  defined  inverse  when 
restricted  to  a  subspace  Hv  =  span  {11(3;)^}^^^  mapping  onto  its  closed  range  Vg(Hv)  = 

WW). 

Proof: 

Clearly  Vg  maps  onto  its  range.  We  show  that  Vg  :  Hv  »->■  Vg(Hv)  is  one  to  one.  Let 
fi>h  £  %v  and  suppose  Vgf\  =  I^/2.  This  means  that  for  all  x  £  Q 

</i,n(x)5)  =  (f2,E(x)g) 


or 

(h  -  f2,n(x)g)  =  0. 

From  this  we  may  conclude  that  (/x  -  /2)  -L  Hv ■  Since  Hv  is  a  linear  subspace  (/x  -  f2)  £  Hv- 
Thus,  fi  —  /2  must  be  zero.  We  have  shown  that  Vg  :  Hv  >->■  Vg{Hv)  is  bijective  and  therefore 
has  a  well  defined  inverse  V~l  :  Vg(Hv)  >-»■  Hv- 


In  specific  cases,  e.g.  wavelet  and  Gabor,  it  can  be  shown  by  approximate  identity  arguments 
[HW89]  that  the  group  representation  transform  Vg  has  a  well  defined  inverse  on  its  range.  In 
fact,  if  /  £  Hv  we  may  formally  write 

/  =  i((V)W.n®)w 

in  L 2.  Figure  4  depicts  the  group  representation  transform  mapping  Vg  and  its  properties  on 
different  domains  and  ranges  of  interest. 

In  light  of  Proposition  4.1.1  and  the  operator  C*  it  is  clear  that  only  functions  which  are 
contained  in  Hv  have  any  hope  of  being  represented  by  linear  combinations  of  {ir(xn)g}.  Thus, 
we  require  that  H  C  Hv- 

To  make  the  connection  between  irregular  sampling  in  Vg(H)  and  discrete  representation  in 
H  fix  Xo  £  Q  and  consider  the  function  II(a:o)<7  G  H.  Since  II(xo )g  is  an  element  in  H  we  may 
compute  its  group  representation  transform  as 


(Fsn(x0)fir)(x)  = 


<n(xo)sr,n(x)0) 

(ff,n(xo1)n(x)pJ> 

(g,H(x~1x)g'j 

(Vg9)(x  o1^)  =  TX0Vgg{x)  =  c2gTXQK(x). 
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Figure  4.1:  The  group  representation  transform  Vg  on  different  domains. 

From  this  calculation  it  follows  that  for  fixed  a  x  £  Q  the  functions  H(x)</  and  c2  TXK  are  group 
representation  transform  pairs.  This  relationship  is  denoted  as 

Vxea,  n(x)sf  ^  c]TxK. 

The  sampling  operator  C  whose  domain  is  Vg(H)  has  an  associated  discretization  operator 
L  whose  domain  is  7 i.  Let  F  =  Vgf.  Then  by  the  reproducing  property  and  Corollary  2.4.2 

F(x„)  =  (F,T,„K)Km  =  (/,n(x„)j) . 

This  suggests  the  discretization  L  :H  i->  i2  (Z)  defined  as 

Lf  =  {F(xn)} 


with  the  adjoint 

L*c  =  J2cnU(xn)g. 

Thus,  questions  about  the  frame  properties  of  the  sequence  {TXnK}  in  L2tt(Q)  may  be  translated 
to  questions  about  the  frame  properties  of  the  sequence  {n(xn)<7}  in  H. 

At  this  point  it  may  prove  fruitful  to  present  an  example.  Lets  take  g  as  the  Dirichlet  kernel 
<7  =  dinQ  and  Q  =  (]R,  +)  having  the  group  representation  rx.  With  these  choices  we  shall  see 
that  the  situation  reduces  to  the  case  of  sampling  in  the  bandlimited  space  PWq.  The  group 
representation  transform  associated  with  these  choices  of  g  and  Q  is 

Vgf  =  (f,E{xn)g)  =  (f,rXnd2  »n)  =  Pof, 

where  Pq  is  the  orthogonal  projection  operator  onto  PWq.  Thus,  Vg  maps  X2(1R)  onto  PWq. 
Now  Hv  =  span  {n(x)<7}  =  span  {r^^n}  =  PWq.  Thus,  Vg  —  Pq  trivially  has  a  well  defined 
inverse  when  considered  as  a  map  Vg  :  Hv  Vg(Hv),  i-e.  Pq  ■  PWq  i-+  PWq,  since  it  is  the 
identity  on  PWq.  It  is  then  evident  that  the  discretization  L  =  C  is 

Lf  =  {(f,TXnd2l rfj)}  =  {(Pnf)(xn)} 
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Object 

Group  Transform  Range 

Group  Transform  Domain 

Space 

Vg(Hv)  =  span  {TXK} 

Hv  =  span  {II(a:)gi} 

Kernel 

K 

9 

Translation 

Tx 

Tx 

Group/Representation 

(0,-),  n(z) 

(IR, +),  Tt 

Atoms 

{TrnK} 

{*•  (xn)g} 

Discretization 

CF  =  {F  *  K{xn)} 

Lf  =  {(f,U(xn)g)} 

Adjoint 

C*c  =  Y2cnTXnK 

L*c=  J2cn7e(xn)g 

Table  4.1:  Group  representation  transform  relations  in  the  generic  case. 


with  the  adjoint 


L\ 


=  £< 


Thus,  with  these  choices  of  g  and  Q  it  is  clear  that  it  is  only  possible  to  reconstruct  signals  from 
their  samples  {f(xn)}  if  they  are  members  of  Hv  =  PWq. 


Object 

Group  Transform  Range 

Group  Transform  Domain 

Space 

Vg{Hv)  =  PWn 

Hv  =  PWn 

Kernel 

di-xn 

Translation 

Tx 

Tx 

Group /Representation 

(IR, +),  Tt 

(IR,  +),  Tt 

Atoms 

{Tx„d2irSl} 

{TXnd2xn) 

Discretization 

Lf={Pnf(*n)} 

Adjoint 

£  C  —  ^2/  Cn'rxnd2irfl 

L  C—  Cn TXn 

Table  4.2:  Group  representation  transform  relations  for  the  case  g  —  d-^n  and  Q  —  (1R,  +). 
The  situation  reduces  to  sampling  in  PWq. 


Similar  tables  may  be  constructed  for  the  wavelet  and  Gabor  cases  where  Q  is  the  affine 
group  Qa  in  the  wavelet  case  and  Q  is  the  Heisenberg  group  Gh  in  the  Gabor  case.  Since  g 
may  be  taken  as  any  admissible  function  (almost  arbitrary)  there  are  many  such  tables  that 
may  be  generated.  For  the  cases  of  the  wavelet  and  Gabor  transforms  we  shall  refer  to  the 
generic  Table  4.1.  In  the  last  sections  of  this  chapter  we  will  look  in  detail  at  three  specific 
cases:  Paley-Wiener,  wavelet,  and  Gabor. 

4.1.2  Weighted  Frames 

Let  {wn}  C  ]R+  be  a  sequence  of  strictly  positive  weights.  In  this  section  we  derive  operator 
theoretic  characterizations  for  weighted  sets 

{wrJi{xn)g} 

to  form  frames  for  Hv  —  span  {n(a;)<7}.  In  other  words,  we  are  interested  in  sufficient  conditions 
on  a  set  T  —  {xn}  C  Q  which  insure  that  the  set  of  functions  {wnIl(xn)g}  is  a  frame  for  the 
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(4.1.4) 


space  Hy.  Recall  that  the  measure  of  a  set  E  C  Q  is 

(m(E)  =  [  dfi(x). 

J  E 

Let  {Vn}  C  0  denote  a  set  of  mutually  disjoint  connected  regions  from  Q,  corresponding  to 
a  set  of  sampling  points  {xn  :  xn  £  Vn}.  It  is  also  required  that  the  regions  {Vn}  cover  all  of  Q, 
i.e.  _ 

1)^  =  5- 

The  sampling  set  {xn}  is  regular  if  there  exists  such  a  sequence  of  mutually  disjoint  connected 
regions  from  Q  and  some  compact  set  V  C  Q  such  that 

Vn,  x~lVn  =  V. 

For  general  irregular  sampling  sets  it  is  required  that  there  is  a  compact  set  V  C  Q  such  that 

[j^'Vn^V. 

In  addition,  to  each  region  Vn  we  associate  the  characteristic  Xn  =  l(Vn)  defined  as  the 
indicator  function  of  the  set  Vn.  Thus  the  set  of  functions  {Xn}  forms  an  orthogonal  partition 
of  L^{Q).  We  will  call  such  {14}  a  disjoint  covering  of  Q  and  such  {xn}  an  admissible  sampling. 

Let  Hy  —  span  {II(a;)<7}  C  L2^(Q)  where  fi  is  the  left  Haar  measure.  Recall  that  Vg{Hv)  is 
a  reproducing  kernel  Hilbert  space  with  kernel  K,  i.e. 

VFeft,  F  =  F*K  =  (F,TxK)L2{g). 

Here,  Tx  is  the  translation  operator  in  Q  and  for  convenience  define  Kx  =  TXK.  The  inner 
product  for  Vg{Hy )  is  inherited  from  L2^(Q). 

Now,  define  the  piecewise  approximation  operator  P  :  Vg(7iv)  L2(Q)  as 

PF  i  £(F,  (1!)X„  (4.1.5) 

which  depends  on  {x„}.  Note  that  its  adjoint,  P*  :  L2(Q)  i->  Vg(7iy)  is 

P*F  =  ^2(F,xn)Li{g  )K*n,  (4-1.6) 

and  of  particular  interest  is  the  fact  that 

VF  e  Vg(Hy),  P* PF  =  £  (F,KXn)Ll{g)  \\Xn\\ll(Q)KXn.  (4.1.7) 

From  this  calculation  it  can  be  seen  that  P*P  is  the  frame  operator  Sw  for  the  weighted 
collection  {p(Vn)KXn}.  This  fact  leads  directly  to  Theorem  4.1.2. 

Theorem  4.1.2  Let  Vg(7iy)  be  a  RKHS  with  reproducing  kernel  K.  Let  {xn}  C  Q  be  an 
admissible  sampling.  Then  set  {p{Vn)KXn}  is  a  frame  for  Vg(7iy)  if  and  only  if  there  exists 
constants  A  >  0  and  B  <  oo  such  that 

VF  €  Vg(Hv),  A\\F\\2Ll(g)  <  \\PF\\2Ll{g)  <  2*||F||£, (0), 

where  PF  =  F(xn) lv„. 
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Proof: 

From  (4.1.7)  we  have 

lirrilljw)  =  =  (f.p-pf)^  =  (r,  . 

Corollary  4.1.3  With  the  hypotheses  of  Theorem  4.1.2  in  place  if 

d  =  ||/-P||L^)<1 

then  {^(Tn)-^®n}  is  a  frarne  for  Vg(Hv)  with  frame  bounds  (1  —  d )2  and  (1  +  d )2. 


Proof:  For  any  F  E  Vg(Hv)  write 

l|r|b„<»  -  rf  IIljici  (  <  Ilf  -  rrilijOT  <  11/  -  =  <i2IIF|li»(5) 

so  that 

weW),  |  IIFIliKc,  -  \\pf\\lw)  |  <  <i||f||li(5) 

leading  to 

VF  e  ^(Wv),  (1  -  d)\\F\\Ll(0)  <  \\PF\\Ll(g)  <  (1  +  d)\\F\\Ll(Q), 

and  the  result  follows  by  invoking  Theorem  4.1.2.  ■ 

The  analog  of  Corollary  4.1.3  in  the  domain  Hv  is  given  below. 

Corollary  4.1.4  With  the  hypotheses  of  Theorem  4.1.2  in  place  if  d  =  ||7-  F||L2(^)  <  1 
then  {//(Fn)II(a:n)<7}  is  a  frame  for  Hv  with  frame  bounds  (1  -  d)2  and  (1  +  d)2. 


The  following  theorem  is  a  slight  generalization  of  the  material  appearing  in  [OS92]. 

Theorem  4.1.5  Let  -{Vn}  be  a  disjoint  covering  of  the  group  Q  generated  by  the  sampling  set 

{®n}  C  Q,  and  V  =  [jx^Vn-  Let  Vg(Hv )  C  L2(G)  be  a  RKHS  having  reproducing  kernel  K 
(fj,  is  the  left  Haar  measure).  Define  the  quantities 

d\  =  supX£v  /  I  Kx  -  K  |  dp 

Jq 

d\  =  supaegYj  /  \Kx(a)~  KXn(cr)\dfi(x). 

n  JVn 

If  d\d2  <  1  then  {p(yn)Ti(xn)g}  is  a  frame  for  Hv- 
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Proof: 

With  P  as  in  4.1.5,  write 

\\(I  -  P)F\\ll(g)  =  \\F-Y,F(*n)x»\\k«» 

=  lC(F-F(«0)x-llijW 

=  j  \Y,(F(X)- F(xn))Xn(x)f  dn(x) 

=  J  \£  (F>K*  ~  *.»>x»(*)f  M*) 

=  l'£\(F,Kx-KXn)\2Xn(x)  d/4*), 

JQ 

where  the  last  step  follows  since  Xn  may  take  only  the  values  1  or  0.  Noting 

{F,  Kx  —  KXn)L2^Q)  <  (f\Kx  -  KXn |2 ,  \KX  -  KXn |2  ^ L2(g) 

and  using  Cauchy-Shwarz  we  further  obtain 

\\(I  -  P)F\\ll(g) 

<  [  £  | \F\KX  -  KXn WX»(*)III^*  -  111 W)M*) 

n 

<  \Kz  -  ^*»l^  ll^llijw  1^*  -  K*n\dF)  Xn (x)dn(x) 

■  ?/,„(  sup  |/G  -  ifxjj  Qf  \KX  -  KXn\d^j  dn{x)  11*112,2(0) 

<  ^  f  sup  f  \KX  -  f  (sup  |/fz  -  *CrJ  J  ll-^llz/S (£?)• 

„  \®ev„  JQ  )  Jvn  \z£Q  ) 

In  order  to  remove  the  dependence  on  n  of  the  first  product  term  in  the  sum  we  make  the 
following  estimate: 


sup 
*ev„  \JQ 


(/  | Kz  -  KXn\d^j 


=  sup  f  f  \TZK  -  TXnR\dfi) 
zev„  \Jg  t 

=  SUV  (  [\Tx;1zK  ~  K\dtJ) 
=  sup  (  f  I TZK  -  K\dfi) 

zex-Wn  \Jg  ' 

<  sup  (  /  \TZK  -K\)  dpi, 
2ey  \Jg  J 


since  Vn,  x~lVn  C  V.  Which  leads  us  to 


sup 
x£v  \Jg 


IIP  -  P )F llii(c)  S 

=  dli\\\f\\\ 
Invoking  Corollary  4.1.4  we  are  done. 


1 

(Jq  |  Kx  -  K  |)  dfi  sup  ?/,„  (I  Kx-KXn\)dn(x) 


ll/ll5 
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Remark  4.1.6  As  pointed  out  in  [OS92]  for  the  case  of  the  discrete  wavelet  transform  these 
estimates  are  not  particularly  good. 


Suppose  H  =  PWq,  which  is  a  RKHS  with  reproducing  kernel 

P-x  7a:^2irn 


and  reproducing  formula 


V/GPWq,  /(*)  =  (f,Txd2nQ) . 

Thus,  PIVq  is  an  example  of  a  space  which  fits  the  hypotheses  of  all  the  above  mentioned 
theorems.  We  could,  in  fact,  invoke  Theorem  4.1.5  and  get  (lousy)  estimates  for  the  frame 
bounds.  Alternatively,  though,  we  may  use  Corollary  4.1.4  directly  along  with  a  result  of 
Grochenig  to  conclude  that  the  weighted  set  {wn  rtnd27rn}  is  a  frame  for  PWq  with  the  foflowing 
estimates  for  the  bounds. 

From  [Gro91]  we  have  that  with  a  sampling  sequence  such  that  2 Tfl  <  1  and  T  —  sup  |tt'+i  — 
t,|  then 


|| /  -  P||  <  2Tft  on  PWn 

where  P  is  the  operator  defined  in  4.1.5  and  the  partition  {V^}  is  the  set  of  intervals  having 
the  midpoints  (ti+i  +t,)/2  as  their  end  points  and  wn  as  their  length.  Invoking  Corollary  4.1.4 
we  obtain  the  result  that  the  set 


{®n  Ttnd2v n} 

is  a  frame  for  PWq  with  frame  bounds  (1  -  2TQ,)2  and  (1  +  27’il)2,  or  equivalently  in  the 
frequency  domain  the  set 

{wn  e-t„} 

is  a  frame  for  L2  [— ft,0]  with  the  same  bounds  (1  -  2 TO)2  and  (1  -f  2 TO.)2. 

4.2  Examples 


As  has  been  seen,  the  discrete  representation  of  signals  as  irregular  samples  of  their  group 
representation  transforms  is  a  complete  characterization  provided  certain  countable  sets  of 
functions  constitute  frames  for  large  enough  spaces.  Owing  to  the  generality  of  the  underlying 
square  integrable  group  representations,  the  flexibility  of  this  approach  to  signal  representation 
is  tremendous.  This  is  manifested  in  the  large  number  of  free  parameters  in  the  representation. 
Each  choice  of  group,  group  representation,  analyzing  function  g ,  and  discrete  lattice  leads  to  a 
discrete  representation.  Roughly  speaking  there  are  as  many  different  discrete  representations 
as  there  are  groups,  group  representations,  discrete  lattices  and  functions  g. 

In  this  thesis  we  shall  restrict  ourselves  to  three  classes  of  discrete  representation.  These 
classes  correspond  to  the  specific  group  representation  transforms  of  (i)  bandlimiting,  (ii)  Gabor 
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transform,  and  (iii)  wavelet  transform.  Each  of  these  classes  is  defined  by  specification  of  the 
group  and  the  group  representation,  i.e.  the  pair  {£/,  II}.  Thus,  within  each  class  there  is  still 
considerable  freedom,  e.g.  the  choice  of  analyzing  function  g  and  the  choice  of  discrete  lattice 
T.  For  complete  characterizations  these  choices  can  not,  of  course,  be  made  arbitrarily.  As 
discussed  in  previous  sections,  the  frame  properties  of  collections 

are  directly  relevant.  If  this  set  were  a  frame  then  the  group  representation  transform  Vgf  of 
a  signal  /  evaluated  at  points  on  the  lattice  T  would  be  a  complete  characterization  of  /,  viz. 
Chapter  3. 

The  corresponding  group/group  representation  pair  for  the  three  special  cases  are 

(i)  bandlimiting:  {(1R,  +),rt}, 

(ii)  Gabor  transform:  {GH,€1rt},  and 

(iii)  wavelet  transform:  {GA,TtDs}, 

where  Qh  and  Qa  are  the  Weyl- Heisenberg  and  affine  groups  respectively  given  in  Examples 
2.3.2  and  2.3.1.  The  common  feature  in  each  of  these  group  representations  is  translation.  In 
each  case  the  group  representation  can  be  factored  as  n  =  Uxrt  where  x  and  t  are  appro¬ 
priate  real  values.  Because  translation  in  time  acts  as  modulation  by  a  complex  exponential 
in  frequency,  the  frame  properties  of  sets  of  complex  exponentials  play  a  fundamental  role  in 
discrete  representations  of  these  types.  The  following  lemma  describes  how  frames  of  complex 
exponentials  together  with  Ux  can  be  used  to  generate  frames  for  large  spaces. 

Lemma  4.2.1  Let  supp  g  =  [a,  b]  where  a  <  b  are  real  numbers.  Also  let  Ux  be  a  bounded 
linear  operator  for  each  real  value  x.  Let  and  {xm }  be  real  sequences.  Suppose  that 

is  a  frame  for  Vm  =  (Uxm(PWa<b)y 

with  bounds  Am  and  Bm.  If  there  are  constants  A  and  B  such  that  for  7  £  1R 
0<A<G(7)  =  EK^nT)|2<^<oo  a.e. 

m 

then  {Ttm,nUXmg}  is  a  frame  for  V  =  (Jm  UXm(PWa,b)  with  bounds  A  (infm  Am)  and 
B  (supm  Bm). 

Proof: 

For  /  G  V  write 

5 2\(f’Ttn>,nUXmg)\ 2  =  E\(f^JUXmgr)f  (Parseval) 

m, n  m,n 

m,n 

<  'EBm\\fWZfr\\2 
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=  Y^Bm  j  \Kl){UXrngm)\2di 

m 

<  ^sup  Bm  )?/  \f(i)\2\(uXmgyX'r)\2dj 

=  (su PBm)  [  |/(T)|2EK^m)|2d7 

<  B  (sup  J  |/(t)|2^7 

=  B  (sup  £m)  ll/ll2 
=  5  ^sup  B^j  ||/||2  (Plancherel) 

The  first  inequality  follows  from  the  assumption  that  {e_tmn}  frames  Vm  and  the  fact  that 
/  (UxmgT  G  Vm  for  each  m.  The  lower  bound  is  analogous.  ■ 


It  is  clear  from  this  lemma  that  for  { TtmnUXmg }  to  be  a  tight  frame  then  the  function  G 
must  be  constant  a.e.  The  following  corollary  addresses  the  tightness  issue. 

Corollary  4.2.2  With  the  assumptions  of  Lemma  4.2.1  {Ttmi„UXmg}  is  a  tight  frame  for 

V  =  Un  UXm{PWa,b)  if  and  only  if  G  is  constant  a.e.  and  for  all  m  the  frame  {e_tmn} 
is  tight  with  Am  —  Bm  =  C. 


In  the  cases  of  the  Gabor  and  wavelet  transform  Lemma  4.2.1  may  be  used  to  show  that 
(under  proper  assumptions)  irregular  lattices  exist  which  generate  frames  for  L2(Kl)  and  //|(1R) 
respectively.  These  results  are  stated  in  Theorems  4.2.9  and  4.2.10. 

4.2.1  Paley-Wiener 

Sampling  in  bandlimited  spaces  has  been  the  subject  of  extensive  study.  Uniform  or  regular 
sampling  in  bandlimited  spaces  has  been  studied  by  Whittaker  [Whil5]  in  1915,  Kotel’nikov 
[Kot33]  in  1933,  Shannon  [Sha49]  in  1949.  The  basic  result  of  these  studies  is  the  widely  known 
classical  sampling  theorem. 

Theorem  4.2.3  (Classical  Sampling  Theorem)  Let  fl  >  0  and  2f IT  <  1.  Then  any  /  £ 
PWq  may  be  reconstructed  from  its  uniform  samples  {/(nT)}  as 

/  =  T  Y  /(^Kr^Trfi, 

where  the  sum  converges  in  L 2  and  is  the  Dirichlet  kernel. 


Thus,  the  classical  sampling  theorem  answers  the  following  question: 

What  conditions  on  the  set  {nT}  allow  the  recovery  of  f  from  {{nT}  ,  f(nT)}? 
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The  condition  that  the  Classical  sampling  theorem  gives  is  that  the  sampling  density  T~l  must 
be  at  least  as  large  as  twice  the  value  of  the  cutoff  frequency  Cl.  i.e.  T-1  >  2 Cl.  The  quantity 
2CI  is  called  the  Nyquist  density. 

Let  us  now  relate  the  classical  sampling  theorem  to  the  general  RKHS  approach  taken  in 
the  begining  of  this  chapter.  In  the  case  of  uniform  sampling  the  pertinent  group  is  (R,  -f )  and 
the  sampling  lattice  is  IY  =  nT.  In  terms  of  the  discrete  representation  and  frame  perspective 
the  pertinent  collection  of  elements  is 


{TVir^jrn}  • 


In  the  case  2TCI  =  1  the  set  of  functions  |^jrnrd2irn}  is  an  orthonormal  basis  for  PWq.  The 
discretization  operator  is 

Lf  =  {(f,TnTd2l rn)} 


with  the  adjoint 


L/C  —  ^  '  C7t  Tnf  d‘2'7rQ  • 


Note  that  the  classical  sampling  theorem  could  be  arrived  at  by  considering  the  frame  properties 
of  the  collection  {Tnxd2l rfi}-  In  particular,  if  2 TCI  —  1  this  set  is  a  tight  exact  frame  for 
PWq  with  bounds  A  -  B  =  2CI,  and  correlation  R  =  2 ClI  ( I  is  the  identity).  In  this  case 
L  =  TI.  Thus,  we  conclude  that  for  any  /  £  PWu 


f  =  L*R'Lf  =  T  L*Lf  =  Tj2f(nT)TnTd27rU. 


4. 2. 1.1  Frames  of  Complex  Exponentials 


Consider  now  more  general  sampling  sets  which  are  not  necessarily  uniformly  spaced.  Ir¬ 
regular  sampling  in  band-limited  spaces  has  been  studied  extensively  by  Paley- Wiener  [PW34], 
Levinson  [Lev40],  Beutler  [Beu61],[Beu66],  and  Yao-Thomas  [YT67].  And  recently  all  of  the 
afore-mentioned  theory  has  been  extended  and  unified  by  Benedetto-Heller  [BH90],  [Hel91], 
and  Benedetto  [Ben92]  in  the  context  of  frames.  In  particular,  much  of  the  theory  is  developed 
in  terms  of  frames  of  complex  exponentials.  Feichtinger  and  Grochenig  [FG92]  have  used  the 
notion  of  weighted  frames,  cf.  Section  4.1.2,  (of  complex  exponentials)  to  achieve  good  conver¬ 
gence  rates  for  iterative  reconstructions  from  irregular  samples  where  gaps  in  the  sampling  set 
are  no  larger  than  the  Nyquist  density. 

Let  us  relate  the  situation  to  the  general  RKHS  approach  at  the  begining  of  the  chapter. 
Suppose  that  r  =  {tn}  is  an  irregular  set  of  points  from  the  real  line  1R  and  /  is  a  signal  which 
is  bandlimited  by  Cl  >  0,  i.e.  /  €  PW&.  For  simplicity  we  shall  again  take  g  =  d2 „ci-  With 
these  choices  the  fundamental  question  is  the  following: 

What  conditions  on  the  set  T  =  allow  the  recovery  of  f  from  {{£„}  ,f(tn)}? 


In  this  case  the  discretization  operator  is 


with  the  adjoint 


Lf  =  {(f,Ttnd2n  n)}  =  {f(tn)} 


L/C  —  ^  ^  CnTind2Tr(l. 
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In  answer  to  the  fundamental  question  we  may  state  that  a  sufficient  condition  on  {fn}  is  that 
the  sequence 

{rtn^27rfi} 

constitute  a  frame  for  PWq.  This  is  because  in  this  case  the  discretization  L  is  a  frame 
representation  making  Proposition  3.4.2  applicable  and  in  fact  we  may  write  for  all  /  6  PWq 

f  =  L*R)Lf. 

The  frame  properties  of  the  set  {rtn  d2irfi}  are  then  of  clear  interest. 

By  Plancherel  and  Parseval  it  is  clear  from  the  definition  of  a  frame,  viz.  3.1.1,  that  a  set 

{4>n}  is  a  frame  for  H  if  and  only  if  j <^„  j  is  a  frame  for  H  =  {/  :  /  6  Tfj.  With  this  notation 

PWq  =  X2  [—ft,  ft].  Because  of  the  basic  Fourier  relation 

(nn^2irnr=  e_t„l[_n,fi], 

the  frame  properties  of  the  the  set  in  PWq  are  the  same  as  the  frame  properties 

of  the  set  j  in  L2[— ft, ft].  Thus,  the  frame  properties  of  such  sets  of  complex 

exponentials  are  of  direct  interest  in  sampling  theory  for  bandlimited  functions. 

By  definition,  viz.  Definition  3.1.1,  a  sequence  constituting  a  frame  for  a  Hilbert  space  must 
reside  in  that  space.  Notationally  we  may  abuse  this  requirement  and  write  that  {e_i„}  is  a 
frame  for  L2  [a,  6]  meaning  that  je_tn  l[a,6]|  is  a  frame  for  L2  [a,  b ].  Here  a  <  b  are  real  numbers. 
It  is  easy  to  see  that  if  {e_tn}  is  a  frame  for  L2  [a,  b]  then  {e_fri }  is  a  frame  for  L2  [a  +  c,  b  +  c]  for 
any  real  c.  In  the  time  domain  this  corresponds  to  the  statement  that  |7-tT,e(a+6)/2d27r[(6-a)/2]} 

is  a  frame  for  PW[a>b]  if  and  only  if  {rfnece(a+6)/2d2,q(b_a)/2]}  is  a  frame  for  P W[a+C<b+C] . 

Sets  of  complex  exponentials  {e_tn}  have  been  studied  by  Duffin  and  Schaeffer  [DS52].  One 
of  their  fundamental  results  is  a  sufficient  condition  on  a  sampling  set  P  =  {tn}  to  generate  a 
frame  of  complex  exponentials  for  L2  [—ft, ft].  That  is,  a  condition  which  insures  that  the  set 
{e_tn}  forms  a  frame  for  L2  [—ft,  ft].  This  condition  is  related  to  the  notion  of  uniform  density. 
Recall  that,  in  general,  a  set  T  =  {fn}  has  uniform  density  Ar  if  the  two  following  conditions 
hold. 

(i)  T  is  uniformly  discrete,  i.e.  there  is  a  d  >  0  so  that  for  all  n  m,  \tn  -  tm\  >  d,  and 

(ii)  | tn  —  n(Ar)-1|  <  L  for  some  constant  L  <  o o. 

Since  \nT  —  mT\  =  |n  — m|T  and  |nT  —  n(l/T)_1|  =  0  a  uniform  sampling  set  IY  with  sampling 
period  T  is  a  uniformly  dense  sampling  set  with  density  ATt  =  l/T. 

Theorem  4.2.4  (Duffin-Schaeffer,  [DS52,  Theorem  I])  If  T  =  {tn}  is  a  uniformly  dense 
sequence  with  uniform  density  Af  >  2ft  >  0  then  {e_tn}  is  a  frame  for  L2  [-ft,  ft]. 


The  following  corollary  is  true  by  the  Fourier  transform  isomorphism  relating  L2  [—ft,  ft] 
and  PWq. 

Corollary  4.2.5  If  T  =  {f„}  is  a  uniformly  dense  sequence  with  uniform  density  Ar  >  2ft  >  0 
then  {nnd27rn}  is  a  frame  for  PWq. 
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It  is  interesting  to  note  that  a  sequence  which  is  uniformly  dense  may  have  arbitrarily  large, 
though  finite,  gaps.  The  largest  such  gap  is  the  parameter  L  in  the  definition  of  a  uniformly 
dense  sequence.  For  a  uniformly  dense  sequence  large  gaps,  however,  must  be  compensated  for 
by  portions  of  high  density  elsewhere  in  the  set.  The  uniform  discreteness  of  such  a  sampling 
set  prevents  it  from  having  any  finite  cluster  points.  Thus  there  can  be  no  interval  in  which  the 
density  of  sampling  points  is  unbounded.  Roughly  speaking  we  can  think  of  a  uniformly  dense 
sequence  of  uniform  density  A  as  a  tesslation  of  the  uniform  sampling  {nA-1}.  By  tesslation 
of  the  set  {nA-1 }  we  mean  a  second  set  {fn}  in  which  each  sampling  point  tn  may  be  identified 
with  a  point  nA-1  through  the  relation  |fn  -  nA-1|  <  L.  Figure  4.1.2  illustrates  a  uniformly 
dense  sequence  derived  as  a  tesslation  of  uniform  sequence  of  density  1  on  a  compact  interval. 


(n-1)  n  (n+1)  (n+2) 


{"} 

•  •  • 


Figure  4.2:  A  uniformly  dense  sequence  generated  as  a  tesslation  of  a  uniform  sequence  on  a 
compact  interval. 

Jaffard  [Jaf91]  has  characterized  sampling  sets  F  =  {tn}  which  generate  frames  of  complex 
exponentials  {e_*„}  for  L 2  [—12,  ft]  in  terms  of  unions  of  uniformly  discrete  and  uniformly  dense 
sampling  sequences. 

Although  we  have  taken  the  analyzing  function  g  to  be  g  —  it  should  be  evident  that 
other  analyzing  functions  can  be  used.  In  this  case  we  are  interested  in  the  frame  properties  of 
the  collection  {rtng}.  This  still  leads  to  consideration  of  the  frame  properties  of  the  complex 
exponentials  since 

( rtngT=  e-tn9- 

For  arbitrary  functions  g  there  is  no  reason  that  we  should  expect  that  this  collection  will  form  a 
frame  for  L 2  [—ft,  ft].  In  fact  such  a  collection  could  only  possibly  form  a  frame  for  Z2(supp  g). 
This  is  because  if  /  is  a  function  such  that  the  support  of  /  is  exclusive  of  the  support  of  g,  i.e. 
supp  /flsuPP  <7  =  0,  then 

=  (f9,  e-tn)  =  0. 

So  that  if  the  essential  support  of  g  does  not  cover  [—ft,  ft]  then  there  are  non-zero  functions 
/  €  L2  [—ft,  ft]  for  which  there  is  no  A  >  0  so  that 

A||/||2  <  |  (f,e-tj)  I2- 

4. 2. 1.2  Exact  frames  for  PWq 
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Definition  4.2.6  (Kadec-Levinson  sequence)  A  real  sequence  {tn}  is  a  Kadec- Levinson 
sequence  if  for  some  ft  >  0  the  condition 


is  satisfied. 


Paley  and  Wiener  first  dealt  with  the  question  of  when  a  set  of  complex  exponentials  {e_<n} 
forms  a  basis  for  the  space  A2  [-ft,  ft].  Levinson  [Lev40]  showed  that  the  constant  1/4  is  the 
smallest  such  constant  which  insures  the  completeness  of  {e_*n}  in  A2  [—ft,  ft].  Later  Kadec 
[Kad64]  gave  a  direct  proof  that  {e_<n}  is  an  exact  frame.  These  results  are  stated  in  Theorem 
4.2.7  which  appears  in  [Ben92,  Theorem  34]. 

Theorem  4.2.7  Suppose  the  sampling  set  {i„}  is  a  Kadec-Levinson  sequence  for  a  given  ft  >  0. 
Then  {e_tn}  is  an  exact  frame  for  A2  [—ft,  ft]. 


A  Kadec-Levinson  sequence  F  =  {fn}  for  ft  >  0  is  uniformly  discrete  with  uniform  density 
Ar  =  2ft.  Clearly,  if  the  sequence  were  uniformly  discrete  the  uniform  density  would  be  2ft. 
To  see  that  a  KL  sequence  is  uniformly  discrete  assume  without  loss  of  generality  that  tm  >  tn 
and  write 


inf  ( tm  tn) 
inf  tm  —  sup  tn 

™  n 

(m  -  i)A  -  {n  +  ^)A  =  (m  -  n  -  ^)A, 

so  that  \tm  —  tn\  >  d  =  | A.  Figure  4.1.2  depicts  sequences  which  are  of  the  Kadec-Levinson 
type.  Each  element  tn  in  the  sequence  is  restricted  to  lie  within  a  region  of  length  1/ (4ft) 
centered  at  nA.  This  region  is  indicated  by  parentheses  in  the  figure. 


t 


n-1 


t 


n+2 


...  1 - (  I  )  (  I  )  (  I  )  (  I  ) - ♦-••• 

(n-1)  A  nA  (n+l)A  (n+2)  A 

Figure  4.3:  A  Kadec-Levinson  sequence  {tn}  for  ft  >  0  where  A  = 


4.2.2  Gabor 

We  have  seen  that  the  Gabor  transform  Gg  is  a  special  case  of  the  group  representation 
transform  where  the  underlying  group  is  the  Weyl-Heisenberg  group  Qh  —  IT  x  1R  X  1R  and  the 
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underlying  representation  is  H}i{z,t,  7)  =  zeyTt.  As  previously  noted  the  Gabor  transform  may 
be  written  as 

Ggf(t,  7)  =  (/,n//(M,7)5)  =  (/,  eyTtg) . 

As  a  discrete  representation  of  a  signal  /  we  shall  consider  an  irregularly  spaced  sampling  set 
T  =  (fn,7m)  at  which  the  Gabor  transform  will  be  evaluated.  The  discrete  Gabor  representation 
of  a  signal  /  €  T2(IR.)  is  then 

Mf)  =  {(tn,lm),Ggf(tn,Jm)}. 

Thus,  for  the  Gabor  transform  the  fundamental  question  is  the  following: 

What  are  conditions  on  the  sampling  set  T  =  (tn,7m)  such  that  an  f  G  X2(1R)  may  be 
recovered  from  knowledge  of  A(/)  =  {(tn,ym),Ggf(tn,ym)}? 

In  keeping  with  our  basic  approach  to  discretization  we  are  led  to  consider  the  frame  prop¬ 
erties  of  collections  of  the  form 

{e-y  mnng} 

which  are  the  same  as  the  frame  properties  for  the  collection 

{nneymg}. 

This  is  because  the  modulation  and  translation  operators  commute  under  magnitude,  i.e. 
\eaTb\  =  |rf,ea|.  Appropriately,  we  call  collections  of  this  form  Gabor  systems  and  such  col¬ 
lections  which  are  frames  Gabor  frames.  If  the  discrete  lattice  {(tn>7m)}  is  regular,  i.e.  there 
are  real  numbers  a  and  b  such  that  (fn,7m)  =  (na,mb)  then  we  refer  to  the  collection 

{^rnaem6flf}  • 

as  a  regular  Gabor  system  and  if  it  is  a  frame  a  regular  Gabor  frame. 

4.2. 2.1  Regular  Gabor  Systems 

There  is  a  wealth  of  theory  associated  with  the  Gabor  transform  Gabor  systems  and  their 
frame  properties  in  the  regular  case.  In  this  section  we  review  some  of  the  regular  Gabor  theory 
in  as  far  as  it  is  pertinent  to  our  general  irregular  outlook. 

For  regular  Gabor  systems  { Tnaemf,g }  there  is  a  characterization  their  frame  properties  for 
i2(IR)  in  terms  of  the  product  ab.  Namely, 

(i)  if  ab  >  1  the  regular  Gabor  system  {Tnaembg}  is  not  a  frame  for  T2(IR)  for  any  choice  of 

g, 

(ii)  if  ab  <  1  then  there  exist  analyzing  functions  g  which  generate  regular  Gabor  frames 
{Tnaembg}  for  T2(1R),  and 

(iii)  ab  =  1  if  and  only  if  {rnaembg}  is  an  exact  regular  Gabor  frame  for  T2(1R). 

The  Balian-Low  theorem  [Bal81],  [Low85],  [Dau90],  viz.  [BHW90]  for  a  mathematically 
sound  proof,  is  an  uncertainty  principle  for  exact  Gabor  frames.  It  says  that  an  exact  Gabor 
frame  can  not  be  generated  from  an  analyzing  function  which  is  well  localized  in  both  time 
and  frequency.  For  a  function  g  €  T2(1R)  we  can  think  of  the  quantity  ||t<7(t)||  as  a  measure 
of  its  localization  in  time.  Similarly,  ||7<7(7)||  can  be  thought  of  as  a  measure  of  its  frequency 
localization. 
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Theorem  4.2.8  (Balian-Low)  Let  g  G  Z2(R).  If  {r„aeTO(,£f}  is  an  exact  frame  for  Z2(1R)  then 

ll*5(*)l!  Ii7fi(7)ll  =  oo- 


4. 2. 2. 2  Irregular  Gabor  Systems 

For  g  compactly  supported,  the  following  theorem  gives  conditions  on  an  irregular  discrete 
lattice  in  the  Weyl-Heisenberg  group  to  give  rise  to  a  Gabor  frame  in  Z2(1R,).  It  is  a  direct 
consequence  of  Lemma  4.2.1.  A  theorem  of  this  form  which  is  due  to  Heller  appears  in  [Hel91]. 
Theorem  4.2.9  With  [a,  6]  CEa  compact  interval  and  g  €  PW^^  with  supp  g  ==  [a,  b]  and 
g  G  Z°°(1R),  suppose  {tn}  is  a  real  sequence  such  that 

{e-tn  }  is  a  frame  for  L2  [a,  b] , 

and  {- ym }  is  a  real  sequence  for  which  there  is  a  d  such  that 

Vm,  0  <  d  <  7m+i  -7 m  <b  -  a. 

Then  {elmTtng}  is  a  frame  for  Z2(1R). 

Proof: 

In  Lemma  4.2.1  let  U  =  f77  =  e7  and  =  tn  so  that 

G  =  S  I(€7m5)12  =  lr-*»5|2- 

m  m 

The  condition  that  jm+\  —7m  <  b  —  a  for  each  m  implies  that  each  term  in  the  sum  has  support 
which  overlaps  with  at  least  one  other.  Thus  there  is  an  A  such  that  G  >  A  >  0  a.e.  on  ]R. 
On  the  other  hand,  the  condition  that  {7 m}  is  uniformly  discrete  together  with  the  facts  that 
g  G  PW[af^  and  g  G  Z°°(1R)  implies  a  B  such  that  G  <  B  <  00  on  IR.  By  Lemma  4.2.1 

Kmrtn5} 

is  a  frame  for  (Jm  eim{P^[a,b])  —  L2(IR)  since  the  supports  of  (e7mp)^  overlap  and  cover  HI. 


We  note  that  [Gro92]  provides  a  similar  theorem  for  irregular  weighted  Gabor  systems,  cf. 
Section  4.1.2,  in  which  conditions  on  a  sampling  set  in  Qh  are  asserted  which  insure  that  the 
weighted  system  is  a  frame  for  Z2(IR).  However,  the  conditions  in  [Gro92]  are  more  restrictive 
than  those  of  Theorem  4.2.9. 

4.2.3  Wavelet 

We  have  seen  that  the  wavelet  transform  Wg  is  a  special  case  of  the  group  representation 
transform  where  the  underlying  group  is  the  affine  group  =  1  x  1R+  and  the  underlying 
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representation  is  Il^t, s)  =  rtDs.  As  previously  noted  the  wavelet  transform  may  be  written 
as 

Wgf(t,  s )  =  {/,  IU(f,  s)g)  =  (/,  rtDsg) . 

As  a  discrete  representation  of  a  signal  /  we  shall  consider  an  irregularly  spaced  sampling 
set  T  =  (tmtn,sm)  at  which  the  wavelet  transform  will  be  evaluated.  The  discrete  wavelet 
representation  of  a  signal  /  6  £2(IR)  is  then 

A(/)  =  )?  5m))}  • 

Thus,  for  the  wavelet  transform  the  fundamental  question  is  the  following: 

What  are  conditions  on  the  sampling  set  F  =  (tm>n,sm)  such  that  an  f  £  L2flR)  may  be 
recovered  from  knowledge  of  A(/)  =  {(tm,n,  sTO),  Wg((tTO,n, sTO))} ? 

In  keeping  with  our  basic  approach  to  discretization  we  are  led  to  consider  the  frame  prop¬ 
erties  of  collections  of  the  form 

{Ttm,n^Sm9}  • 

Collections  of  this  form  are  called  wavelet  systems  and  such  collections  which  are  frames  wavelet 
frames.  If  the  discrete  lattice  {((tm)n,am)}  is  regular,  i.e.  there  are  real  numbers  a  and  b  such 
that  (tmi„,aTO)  =  ( a~mnb,am )  then  we  refer  to  the  collection 

{ra-mnbDct™g}  • 

as  a  regular  wavelet  system  and  if  it  is  a  frame  a  regular  wavelet  frame. 

4. 2. 3.1  Regular  Wavelet  Systems 

Recently  regular  wavelet  systems  have  received  an  enormous  amount  attention.  In  fact,  there 
are  seminal  developments  of  wavelet  theory  by  Daubechies  [Dau92],  Mallat  [Mal89c],  [Mal89b], 
[Mal89a],  and  Meyer  [Mey90].  The  material  in  [Dau92]  serves  as  an  excellent  backdrop  to  the 
recent  developments  in  wavelet  theory. 

One  particular  area  of  interest  is  the  existence  and  construction  of  orthonormal  wavelet 
bases.  In  this  regard  the  development  of  multi-resolution  [Mal89c]  analysis  (MRA)  has  been 
a  major  advance.  A  further  development  [Dau88]  has  been  the  construction  of  orthonormal 
wavelet  bases  with  compact  support.  As  a  result  very  fast  algorithms,  e.g.  the  fast  wavelet 
transform,  for  the  computation  of  the  wavelet  representation  have  been  constructed. 

Although  such  bases  certainly  lead  to  invertible  discrete  represenations  with  fast  algorithms, 
we  shall  not  consider  such  special  cases  in  too  much  detail.  For  one  thing,  orthonormal  bases 
preclude  the  benefits  associated  with  redundant  systems  as  discussed  in  Section  3.5.1.  For 
another  thing,  in  many  applications  the  condition  that  the  wavelet  system  must  be  regular  is 
undesirable.  For  instance  in  pattern  classification  a  desirable  property  of  a  representation  of  a 
signal  is  that  it  be  translation  invariant.  In  other  words,  translated  versions  of  signals  should 
yield  translated  versions  of  their  discrete  representations.  This  is  clearly  not  satisfied  for  fixed 
sampling  geometries  such  as  are  associated  with  regular  wavelet  systems.  Moreover,  there  are 
still  reasonably  fast  algorithms  for  the  computation  of  the  wavelet  representation  for  irregular 
systems. 
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4. 2. 3. 2  Irregular  Wavelet  Systems 

In  answer  to  the  fundamental  question,  Theorem  4.2.10  gives  conditions  on  a  sampling  set 
in  Ga  to  give  rise  to  a  wavelet  frame  for  'H^.(IR).  It  is  a  direct  consequence  of  Lemma  4.2.1. 
Theorem  4.2.10  Let  0  <  a  <  b  be  two  strictly  positive  real  numbers  and  let  g  G  /W[a ;,]  and 
g  G  L°°(]R)  with  supp  g  =  [a, 6]. 

Suppose  that  Q  Ht  and  {sm}  C  IR+  are  two  sequences  such  that 

ie-tm,n}  is  a  frame  for  L2  [sma,sm6] 


and  there  is  a  d  such  that 


Then 


Vm,  1  <  d  <  — 7^-1-  <  — . 

Sm  O' 

{Ttm,nDsmg}  is  a  frame  for  tf+(IR). 


Proof: 

In  Lemma  4.2.1  let  U  =  Us  =  Ds  so  that 

g  =  D(i>.„9n2  =  £ 

m  m 

Since  the  support  of  D  -ig  is  Im  =  [sma,  sm6],  the  condition  on  {sm}  that  <  smb 

insures  that  Im  and  Im+i  have  some  overlap  and  are  not  identical.  Since  g  G  L°°(1R)  we  may 
conclude  that  there  are  constants  A  and  B  so  that  0<yl<G</l<oo.  Clearly  [J  lrn  =  (0,  oo) 
so  that  V  =  Um  ^sm(PW’[a  b])  =  PW( 0,oo)  =  H+0&)  since  identically  constant  functions  are  not 
in  L2(1R).  Thus,  by  Lemma  4.2.1  {rtm  nDSmg}  is  a  frame  for  P2(1R).  ■ 


Note  that  an  identical  theorem  which  concludes  that  {Ttm  nDSmg)  is  a  frame  for  L2(1R)  can 
be  arrived  at  by  requiring  the  analyzing  function  g  in  Theorem  4.2.10  to  be  even.  Finally,  we 
note  that  as  in  the  Gabor  case  [Gro92]  provides  a  similar  theorem  for  irregular  weighted  wavelet 
systems,  cf.  Section  4.1.2,  in  which  conditions  on  a  sampling  set  in  Ga  are  asserted  which  insure 
that  the  weighted  system  is  a  frame  for  L2(1R).  However,  the  conditions  in  [Gro92]  are  more 
restrictive  than  those  of  Theorem  4.2.10. 
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Chapter  5 

Local  Frames 


In  previous  chapters  we  have  seen  how  it  is  possible  to  represent  signals  from  an  infinite 
dimensional  Hilbert  space  using  frames  generated  from  irregular  samplings  of  certain  reproduc¬ 
ing  kernel  Hilbert  spaces.  Because  these  are  frame  representations  it  follows  that  every  signal 
in  the  Hilbert  space  can  be  reconstructed  from  its  frame  representation.  With  respect  to  the 
situation  in  which  it  is  of  interest  to  reconstruct  only  a  specific  single  function  in  the  Hilbert 
space,  a  full  frame  representation  is  much  more  than  is  needed.  Moreover,  even  though  frame 
representations  are  discrete,  storage  and  manipulation  on  a  digital  machine,  require  not  only  a 
countable  representation  but  also  a  finite  representation.  Both  of  these  observations  motivate 
the  notion  of  a  local  frame.  The  chapter  concludes  with  a  scheme  for  signal  compression  using 
local  frames. 

5.1  Frame  Localization 


Suppose  7i  is  a  Hilbert  space  of  interest.  On  the  one  hand  the  global  theory  of  frames,  viz. 
Chapter  3,  allows  the  reconstruction  of  every  signal  /  £  7f  from  its  frame  representation.  On 
the  other  hand  the  fundamental  goal  of  discretization  theory  is  to  reconstruct  only  a  particular 
signal  /*  €  H  from  its  discrete  representation.  Although  the  global  frame  representation  is  a 
viable  discrete  representation  which  meets  the  mandate  of  the  discretization  theory,  its  ability 
to  recover  every  signal  is  far  more  than  is  required.  Since  it  is  not  necessary  to  reconstruct 
every  signal  in  the  entire  Hilbert  space  H  it  is  natural  to  ask  if  there  is  some  method  in  which 
the  global  frame  representation  may  be  localized  about  a  particular  signal.  These  ideas  lead 
directly  to  the  notion  of  a  local  frame. 

The  distinguishing  feature  of  a  local  frame  for  H  is  that  it  is  dependent  on  a  particular 
signal  in  H.  Thus,  for  every  signal  in  7i  there  is  an  associated  local  frame.  In  this  section 
we  examine  two  methods  for  frame  localization:  one  entailing  the  signal  dependent  choice  of 
frame  elements,  i.e.  signal  dependent  sampling,  and  the  other  entailing  the  finite  truncation 
of  frame  representations.  The  first  method  of  localization  involves  the  construction  of  a  global 
frame  which  has  elements  which  are  signal  dependent.  With  respect  to  the  frame  {H (x)g}x^r 
localizations  of  this  form  will  come  from  sampling  sets  T(/)  which  are  signal  dependent.  The 
second  method  of  frame  localization  is  via  truncation  of  a  global  frame.  Truncation  has  the 
desirable  property  that  it  necessarily  results  in  a  finite  discrete  representation  of  a  signal. 
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Localization  can  be  viewed  in  terms  of  a  decomposition  of  the  Hilbert  space  H  into  two 
signal  dependent  subspaces.  If  /*  £  H  is  the  particular  signal  of  interest,  localization  results  in 
a  decomposition  of  TL  as 

H  =  W(/,)©7f(/*)x 

where  /»  is  (almost)  contained  in  the  finite  dimensional  subspace  TL(f*). 

In  the  following  sections  it  is  assumed  that  TL  is  a  Hilbert  space  contained  in  Z/2(H)  and 
Q  is  a  group  with  representation  n  on  TL .  Further  g  is  assumed  to  be  an  admissible  function, 
i.e.  g  €  Autfl(TL)  where  fx  is  the  left  Haar  measure.  Recall  that  the  generic  representation  of  a 
signal  /  G  TL  is 

«/,n(z)<7>W 

where  T  C  Q  is  a  countable  sampling  set  from  Q.  This  representation  may  be  thought  of  as  the 
irregular  sampling  of  the  group  representation  transform  Vg  :  TL  Lft(G)  given  in  Equation 
(2.4.1). 

5.1.1  Signal  Dependent  Sampling 

In  previous  discussions  we  have  not  addressed  the  issue  of  how  the  sampling  set  T  arises. 
From  a  conventional  point  of  view  T  would  be  considered  to  be  fixed  and  given.  Here,  instead 
we  view  the  sampling  set  T  as  a  design  parameter  for  signal  representation.  In  particular,  we 
may  adapt  the  sampling  set  to  fit  particular  signals  of  interest.  As  a  result,  we  deal  with  signal 
dependent  sampling  sets  T(/)  and  study  the  frame  properties  of  sequences 

{n(a:)5}a,Gr(/j 

for  particular  signals  /  6  TL .  In  general  we  look  for  signal  dependent  samplings  F(/)  C  T  which 
are  embedded  in  global  samplings  F  which  generate  frames  for  the  whole  space  TL.  This  is  one 
method  by  which  we  may  achieve  frame  localization. 

As  an  example  of  a  frame  generated  by  a  signal  dependent  sampling  let’s  look  at  the  familiar 
case  of  sampling  in  the  bandlimited  space  TL  —  PWq.  In  this  case,  viz.  Section  4.2,  with 
g  =  dinti  the  sampling  set  T  =  {t„}  is  a  sequence  of  real  numbers  and  the  representation  of 
/  €  PWq  is  {f(tn)}.  Suppose  that  /»  6  PWn  is  the  particular  non-zero  signal  of  interest. 
Define  the  k- th  moment  sampling  set  r^(/*)  of  /*  as 

r  <*>(/,)  =  {t:/ifc)(t)  =  o}, 

where  /f^  is  the  k-th  derivative  of  /*.  Being  in  PWq,  /*  is  analytic  and  the  moment  sampling 
sets  |r(fc)(/*) |  are  all  well  defined  and  countable.  A  nice  feature  of  the  moment  sampling  sets 
is  that  they  are  translation  invariant,  i.e.  with  a  6  1R 

vfc,  r<*)(T,/.)  =  rar<*>(/*). 

According  to  the  Duffin-Schaeffer  theorem  (Theorem  4.2.4)  for  {e_t}<er  to  form  a  frame  for 
PWq  it  is  needed  that  T  have  uniform  density  (Section  2.5.2)  AT  >  2fi.  We  state  without 
proof  that  such  a  sampling  set  may  be  generated  as  a  finite  union  of  moment  sampling  sets. 
That  is,  we  construct  a  signal  dependent  sampling  set  F(/*)  as 

r(/.)  =  U  r(i)(/>). 

keF 
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where  FC  Z  is  a  finite  set.  The  reader  is  referred  to  Section  6.2.1  for  a  numerical  example  of 
such  a  sampling  scheme  and  its  associated  reconstructions. 

5.1.2  Truncation 


So  far,  our  approach  to  discretization  has  resulted  in  representations  of  signals  which  are 
countable  yet  possibly  infinite.  For  practical  reasons  it  is  also  necessary  to  have  representations 
which  are  finite.  A  natural  procedure  for  going  from  a  countably  infinite  representation  to  a 
finite  representation  is  truncation.  Such  a  truncation  is  necessarily  signal  dependent  since  it  is 
the  infinite  representation  of  the  signal  on  which  the  truncation  is  performed.  In  this  section 
we  shall  assume  that  {n(z)<7}xer  is  a  frame  for  H  with  frame  representation  L.  If  T  —  {zn} 
then  L  is  given  as 

Lf  =  {Vgf(xn)} 

with  the  adjoint 

L*e  =  ^CnJl(xn)g. 

Define 

Us{f*)  =  span  {II(zn)p  :  j  (/,  U(xn)g)  \  >  0}  (5.1.1) 

We  think  of  the  space  as  the  localized  space  around  the  signal  /»  with  respect  to  the 

frame  {II(zn)g}.  To  further  develop  this  idea  we  introduce  the  notion  of  a  frame  truncation 
operator. 


Example  5.1.1  Let  JC  Z.  For  localization  around  a  specific  /*  the  set  J  will  depend  on  /*. 
A  simple  frame  truncation  operator,  F  :  t2  (2)  i-»- 12  (J)  is 


(Fc)n 


cn ,  n  £  J 
0,  otherwise 


(5.1.2) 


where  c  £  l2  (2).  This  F  has  the  following  properties: 

a.  F  is  a  linear  operator, 

b.  ||Fj|  =  1, 

c.  F  =  F*  is  self  adjoint, 

d.  F  =  F2,  and 

e.  F  is  the  orthogonal  projection  operator  onto  the  subspace  F2(J). 


Proof: 


a.  With  x,y  elements  of  H  and  a,/3  complex  scalars,  clearly,  F(ax  +  (iy)  —  aFx  +  (3Fy. 

b.  (i)  imi  >  i 

Pick  a  j*  £  J  and  let  c  =  Then  we  have 

||F||  =  sup  ||jy||  >  \\Fc\\  =  1.  (5.1.3) 

\\d\\=i 
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(a)  Ml  <  i 


Mil2  =  D*/ i  5>l2  =  IMI2- 

jeJ 

c. 

< Fc’c )  =  J2(Fc)j^  =  12  N2 

ja 

(C,fc)  =  EcMi  =  EN2 

jeJ 

d.  Trivially,  F2x  =  F(Fx)  =  Fx. 

e.  By  Fact  2.1.2  items  a.  through  d.  imply  e. 


Example  5.1.2  As  a  more  specific  example  of  a  frame  truncation  operator  consider  the  fol¬ 
lowing  F  defined  with  respect  to  a  specific  /*  £  H  and  8  >  0.  A  family  of  truncations  {Ffmis} 
is  given  as 

|(c*)n|  >  8 
otherwise, 

x 

where  c*  =  A/* 


(Ff.,sc)n 


{cni 

0, 


(5.1.4) 


For  all  c  £  L(7i)  and  8  >  0,  the  truncation  F/,}$  provides  an  orthogonal  decomposition  of  c 
as 


c  =  Ff*,sc+  {I  ~  FU,s)c 


and 


l|c||2  =  ||F/.,5c||2  +  ||(/-F/.,5)c||2. 

Such  an  Fftis  partitions  c  into  two  segments:  one  for  which  c*  has  elements  larger  than  8  and 
one  for  which  c*  has  elements  less  than  or  equal  to  8.  The  two  following  lemmas  show  that 
(i)  the  former  segment  resides  in  a  finite  dimensional  space  and  (ii)  it  is  always  possible  to 
determine  a  8  which  will  ensure  that  an  arbitrary  percentage  of  the  energy  from  the  whole 
sequence  c*  will  be  contained  in  this  first  finite  dimensional  segment. 

Lemma  5.1.3  Suppose  Ffmj  is  as  in  Example  5.1.2  for  a  fixed  /*  £  H.  For  all  8  >  0 

dim  {F}tjL(H)}  <  oo. 


Proof: 
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We  have 


I|C,||!  >  ||F/.,5C.||  =  ■£  |(c.)n|2 

>  82  card  {n  :  |(c*)„|  >  8} 

=  62  dim  {Ffti$L(H)} , 

so  that 

dim  {Ff„sL(H)}  <  <  oo 

since  c*  £  l2  (Z)  and  8  >  0.  ■ 

Lemma  5.1.4  Suppose  Ffmy$  is  as  in  Example  5.1.2.  Given  e  >  0  there  is  a  8  so  that 

\\(I  -  FfttS)c,\\2  <  £||c»||2.  (5.1.5) 


Proof: 

Clearly, 


lim  HF/.^c.ll2  = 


<  oo. 


Therefore,  for  e  arbitrary  there  is  some  8  >  0  so  that 

|||c,||2  -  ||E>.,5c*||2|  <  e||c,||2. 

Since  ||c*||2  =  ||E/,,«c*||2  +  || (I  -  E/.,s)c»|[2  we  may  conclude  ||(/  -  Fftls)c* ||2  <  c||c»||2.  ■ 


Equation  (5.1.5)  expresses  the  notion  that  the  operator  FfttgL  extracts  the  most  significant 
frame  coefficients  with  respect  to  the  specific  signal  /».  Here  the  term  “most  significant”  is 
quantified  by  the  parameter  e  £  (0,1).  For  example,  a  value  of  f  «  0  indicates  that  almost 
every  coefficient  is  significant,  and  a  value  of  e  &  1  indicates  that  almost  every  coefficient  is 
insignificant.  Via  this  lemma  there  is  an  interplay  between  the  specified  value  of  e  and  8.  In 
fact,  Lemma  5.1.4  implies  the  existence  of  a  truncation  distribution  function  i/(e)  which  serves 
as  the  boundary  between  acceptable  and  non-acceptable  thresholds  8  for  a  given  t.  Given  a 
particular  /*  £  ft  define  the  truncation  distribution  function  vj ,  :  (0, 1)  <->■  [0,  ||c„||oo]  associated 
with  the  frame  representation  L  as 

"M  =  inf  {*  :  IK7  -  Ff.,s)c* II2  <  e||c,||2}  ,  (5.1.6) 

where  c*  =  L/*.  A  possible  truncation  distribution  function  is  shown  in  Figure  5.1.  Proposition 
5.1.5  asserts  that  a  truncation  distribution  function  must  be  monotonically  increasing. 
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Proposition  5.1.5  Given  a  signal  /„  £  W,  a  truncation  distribution  function  v/,  as  defined  in 
(5.1.6)  is  a  monotonically  increasing  function  which  is  continuous  from  the  left  and 

limi'f.(e)  =  0 

lrni  !//.(£)  =  HcJloo- 


Proof: 

We  show  v  is  monotonically  increasing.  Let  ei  <  e2  and  define  the  sets  S\  and  S2 
Si  ±  {S  :  ||(/-F/.,5)c*||2<ct||c,||2},  *  =  1,2. 

Clearly  Si  C  S2  so  that  inf  <  inf  S2  and  consequently  ^/,(ci)  <  vjt (e2).  ■ 


8 


l  e 


Figure  5.1:  A  possible  truncation  distribution  function  u(e) 

The  truncation  distribution  describes  the  relation  between  the  necessary  value  for  the  trun¬ 
cation  threshold  6  and  the  desired  percentage  of  energy  preservation  e  required  after  truncation. 
Typically,  a  value  of  e  is  prescribed  from  which  a  compatible  threshold  S  is  computed  via  the 
truncation  distribution  v ,  i.e.  6  =  i'(e).  Suppose  /*  is  a  particular  signal  in  H,  L  is  the  pertinent 
frame  representation  and  e  is  chosen  as  a  fixed  value  between  0  and  1.  With  such  a  prescribed 
e,  if  6  =  is(e)  it  is  assured  that 

\\(I  -  Ff.tS)Lf4*  <  e\\LU\\. 

To  see  the  ramifications  of  this  requirement  in  the  signal  domain  H  let  us  first  introduce  the 
concept  of  “essential  containment”. 

Definition  5.1.6  A  signal  /  €  H  is  e-contained  in  a  subspace  H'  C  H  if 

||(/-^)/l|2<^ll/H2, 

where  P-^i  denoted  the  orthogonal  projection  operator  onto  the  subspace  H1 ,  and  we  may  write 
f£H'  by  e. 
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By  the  previous  discussion  if  S  =  u(e)  we  may  say  that  Ff,tgLf *  is  c-contained  in  Ffm  gL(H). 
Moreover,  the  essential  containment  property  can  be  related  back  to  the  signal  domain  by 
Theorem  5.2.1. 


5.2  Finite  Representation 


In  this  section  we  examine  localized  frames  resulting  from  truncation  of  a  global  frame  for 
a  Hilbert  space  H.  The  truncation  process  results  in  a  finite  dimensional  subspace  of  'H  which 
itself  is  framed  by  finitely  many  elements  taken  from  the  global  frame.  Being  a  frame  for 
the  localized  space,  the  finite  dimensional  frame  has  an  associated  local  frame  representation. 
Reconstruction  from  local  frame  representations  are  examined  and  bounds  for  the  corresponding 
error  are  developed. 

5.2.1  Local  Frame  Representation 

Assume  that  {</>„}  is  a  global  frame  for  a  Hilbert  space  H  C  L2(\\l)  with  frame  representation 
L  and  frame  correlation  R.  In  relation  to  previous  material  we  may  have  <f>n  —  U(xn)g,  but  this 
is  not  necessary  for  the  discussion  here.  Fix  6  >  0  and  consider  a  particular  element  /*  G  H 
and  the  localization  associated  with  the  truncation  operator  Fftj  given  in  Example  5.1.2.  From 
Lemma  5.1.3  we  see  that  localization  by  truncation  has  the  property  that  the  truncated  space 
7fs(/»)  is  finite  dimensional.  Here 

Hs{U)  =  span  {<f>n  :  >  8} 

=  span  {</>„  :  n  G 

=  Ff„sL(H ), 

where  «/$(/*)  =  {n  :  \  ( f*,4>n )  \  >  <S}  and  card /«(/*)  <  oo. 

Because  any  finite  collection  of  functions  is  a  frame  for  its  span  [Pat92]  we  conclude  that 

is  a  frame  for  Moreover  the  associated  frame  representation  operator  with  respect  to 

the  truncated  frame  is  L/,,s  =  FftjgL  with  local  frame  correlation  Rf,,s  =  FfmtgRFftts.  A  local 
reconstruction  fs  starting  from  an  arbitrary  /  G  H  is 

h  =  L*^sR)^Lu<sf 

=  (Fftt5L)*(Fu,sRFu^Ff,tSLf 
=  L*FlsR'FlsLf 
=  L*Fu,5R'Fu,sLf 

because  Fftt$  is  an  orthogonal  projection,  viz.  Example  5.1.1.  Thus,  a  local  reconstruction 
may  be  thought  of  in  terms  of  the  truncation  of  the  correlation  matrix  Rftls-  The  L2-error 
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associated  with  the  local  reconstruction  is 


II/. -All  =  Itrfi'Lf,  - 

=  \\L'R'(l-F,,f)Lf,\\ 

<  lli'll  11**11  W  ~  F,.,s)Lf.\\ 

<  ~  Ft.^Lf.W- 

Thus,  if  Fj,jLf*  is  c-contained  in  Ff,fL(H)  then  fs  is  ^-c-contained  in  Hg(f »)  where 
is  given  in  (5.1.1).  In  fact,  Theorem  5.2.1  improves  on  this  result  with  a  tighter  bound  on  the 
essential  inclusion  of  ®  instead  of  jjy. 

Theorem  5.2.1  provides  controllable  error  bounds  on  the  local  frame  representation  of  a 
signal.  More  than  this,  it  provides  a  precise  statement  of  the  notion  that  a  signal  can  be  well 
represented  by  the  most  important  (e.g.  largest)  coefficients  in  its  frame  expansion  and  implies 
a  natural  decomposition  of  the  space  H  as 

Theorem  5.2.1  Given  a  signal  /*  6  H  ,  suppose  is  a  frame  for  7 i  with  operator  S , 
representation  operator  L,  frame  correlation  R,  and  frame  bounds  A  and  B.  Given  e  >  0,  if 
8  =  i/(e)  then 

II/.  ~  Ml2  r  ,  B 
ll/.ll2  A’ 

where 

fs  =  L*(Fu,5tfFu,s)LU  =  S-xL*Fh,sLU. 

Proof: 

First,  we  establish  the  formal  identity  S~1L*Fj,ysLfar  =  L*(Fj,tsR*Ff.,s)Lf*.  We  have 

=  S-\Fh,sLy{Fh,sL)U 
=  5>(J-A  L*Ly(Fft,sir(Ff„sL)f* 

=  ^F!^Ly{I-\LLy{Ff„sL)U 
=  {Ff^LYR\Fu,sL)U  =  L\Ff^R]Fu,s)LU. 

Now,  write 

/.-/«  =  S~XSU  -  S~1L*Ff,tgLf*  =  S~lL*{I  -  FfttS)LU  (5.2.1) 

Because  S  is  a  frame  operator  we  have  that 

VjeW,  A\\gf  <  (Sg,g)  <  B\\g\\2.  (5.2.2) 

In  particular 

A||/* -Ml2  <  {L\I-Fu,s)Lf*,S-lL*{I-Fs„s)LU) 

=  {(I  ~  Ff.,s)Lf*,LS~lL*(I  -  Ff„s)LU ) 

<  ||(/  -  Fj.,s)LU ||  \\LS~X L*{I  -  FS.,S)LU II 
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<  \\LS-H*\\\\(I  -  Ff.,s)Lf42 

<  \\{I-Fu,s)LU\\2 

<  4Lf*\\2 

<  eB\\f*\\2, 

from  which  the  result  follows.  The  manipulations  are  justified  respectively  as  frame  definition, 
adjoint  operator  property,  Cauchy-Schwarz  (and  the  fact  that  (/*,£/*)  is  real  and  positive, 
i.e.  S  is  a  positive  real  operator),  operator  norm  inequality,  ||I/S,-1Z/*||  <  1  (by  Fact  2.1.2 
LS~lL*  is  the  orthogonal  projection  onto  the  range  of  L ),  application  of  Lemma  5.1.4,  and 
finally  ||L||2  <  B.  ■ 


Theorem  5.2.1  shows  that  with 

Hstf*)  =  span  {<f>n  :  \(f,,<f)n)\  >  S}  =  Ffm}SL(H) 

we  have 

by  ef , 

where  6  =  i/(e).  We  have  decomposed  the  space  H  as 

n  =  HsU*)  ©  W.)\ 

where  /*eH«(/*)  and  /*i7f5(/*)-L. 


5.2.2  Local  Frame  Correlation 

In  Section  3.3  we  examined  the  properties  of  a  general  infinite  dimensional  frame  correlation 
R.  There  we  saw  in  Proposition  3.3.3a.  that  an  infinite  dimensional  frame  correlation  R 
is  not  compact.  Local  frame  correlations,  however,  are  finite  dimensional  and  therefore  are 
compact.  This  implies  the  existence  of  eigenvalues  and  allows  the  incorporation  of  standard 
matrix  techniques  such  as  singular  value  decompositions  to  expose  the  eigen-structure  of  a  local 
frame  correlation  matrix.  From  Proposition  3.3.2  local  frame  correlations  Rf„s  =  Ff,,sRFf,,s 
are  matrices  given  explicitly  as 


Bf*,S  ~  ( (/>m,n)m,n£Js{f *) 


where 

Hf*)  =  {n  :  |  (/*,<£„)  I  >*)• 

The  following  theorem  relates  the  frame  bounds  of  a  local  frame  to  the  eigenvalues  of 
the  local  frame  correlation  matrix.  In  particular  it  shows  that  the  maximum  and  minimum 
eigenvalues  associated  with  eigenvectors  in  the  range  L(7i)  are  the  values  of  the  frame  bounds. 
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Theorem  5.2.2  Let  {4>n}n^j  be  a  local  frame  for  the  finite  dimensional  Hilbert  Space  H  — 
with  local  best  frame  bounds  A  and  B  and  frame  correlation  Rjtt5-  Then  the  frame 
correlation  R  =  Rftts  is  related  to  the  local  frame  bounds  A  and  B  as 

A  =  min  ai(R ) 

B  =  max  ctl(R), 

where  crjj(-)  denotes  the  spectrum  restricted  to  the  range  L(H). 

Proof: 

This  may  be  proven  as  a  corollary  to  Theorem  3.3.6;  however,  we  give  the  following  direct 
proof.  We  shall  show  that  B  =  maxox(.ft).  The  proof  of  the  lower  limit  A  —  min  cri(R)  is 
analogous. 


(i)  max<j£,(.R)  <  B 

Since  {<j>n}  is  a  frame  for  H  then  L*  is  surjective.  Hence,  for  all  /  6  H  there  is  a  c  6  t2  (Z) 
so  that  f  —  L*  c  and  by  the  frame  property 

Vc  €  L(H)  A\\L*c\\ 2  <  (L*c,SL*c)  <  H||L*c||2, 

where  S  =  L*L  is  the  frame  operator.  This  is  equivalent  to 


and  so 


Vc  G  L(H)  A  (c,  LL*c)  <  ( L*c ,  L*LL*c)  <  B  (c,  LL*c) , 


Letting  A  be  and  eigenvalue  of  R  in  L(H),  i.e.  for  some  c  G  L(H),  Rc  =  Ac,  we  have 

A  ~  A[|c|P  ~  A  -  B' 

Therefore,  in  particular,  maxo/^i?,)  <  B. 


(ii)  max  <?l(R)  >  B 

Now  let  C{  be  the  eigenvector  associated  with  the  eigenvalue  A;  of  R.  Since  span  {c,}  =  L(H) 
then  any  c  G  L(H)  may  be  written  as  c  =  Y^aici  f°r  some  complex  sequence  {cvj}.  For  all 
c  G  L(H), 


< 


< 


so  that  B 


A 


supc 


( c,R2c ) 
{c,Rc) 


<  max  Uh^R). 


£Z)(ci,^2Cj) 

i  j 


*  3 


(maxAJ)^J]Aj{ct,cJ) 
«'  j 

max(TL(R)  (c,  Rc) , 
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5.3  Compression 

Compression  of  data  is  a  natural  goal  in  applications  which  seek  to  minimize  the  storage 
or  medium  capacity  needed  to  hold  the  information  contained  in  a  signal.  In  this  section  we 
describe  a  general  method  for  data  compression  using  local  frames.  The  method  is  constructed 
so  that  a  local  representation  of  an  arbitrary  signal  satisfies  a  prescribed  information  constraint. 
The  information  constraint  is  assumed  to  be  a  fixed  bit  rate  br  (bits  per  second).  Accordingly, 
compression  comes  from  the  quantization  of  local  frame  representations  in  such  a  way  as  to 
meet  the  information  constraint.  This  compression  method  results  in  a  coding  scheme  which 
is  hierarchical  in  the  sense  that  the  code  generated  by  the  compression  method  with  a  low 
bit  rate  constraint  is  embedded  in  a  code  generated  at  higher  bit  rate  constraint.  Numerical 
reconstructions  and  performance  evaluation  of  the  compression  method  are  detailed  in  Section 
6.3.2. 


5.3.1  Compression  Ratios 

When  dealing  with  data  and  schemes  for  data  compression  it  is  natural  to  introduce  a 
measure  of  compression.  In  simple  terms,  a  “compression  ratio”  measures  the  relative  decrease 
in  complexity  of  data  in  a  raw  form  as  compared  to  the  complexity  of  its  new  compressed  form. 

For  speech  it  is  customary  to  deal  directly  with  bit  rates  instead  of  compression  ratios.  This 
is  because  the  bandwidth  of  speech  is  a  fixed  constant  which,  for  some  practical  purposes,  may  be 
taken  to  be  il  =  4000  Hz.  Consider  the  “raw”  form  of  an  analog  speech  signal  /*  to  be  a  sampled 
version  with  8  bits  per  sample  and  a  uniform  sampling  period  of  T  =  1  / (20)  =  1/8000  seconds. 
In  this  case  the  required  “raw”  bit  rate  is  64Kbps.  Since  the  hypotheses  of  the  classical  sampling 
theorem  are  satisfied,  it  is  possible  to  reconstruct  (modulo  slight  errors  due  to  quantization)  the 
original  speech  signal  /*.  Any  representation  which  allows  for  recovery  of  the  original  speech 
signal  /*  and  requires  a  bit  rate  less  than  64Kbps  is  a  compressed  version  of  /*.  Consequently, 
a  compression  ratio  of  10:1  will  be  achieved  by  a  particular  speech  compression  scheme  if  that 
scheme  yields  a  bit  rate  of  6.4Kbps. 

It  is  clear  that  a  similar  calculation  can  be  made  for  signals  from  an  arbitrary  bandlimited 
space  (with  a  bandlimit  other  than  4KHz).  Since  in  practice  we  may  consider  all  signals  of 
interest  to  be  bandlimited,  we  deal  directly  with  bit  rates  instead  of  compression  ratios. 


5.3.2  Approach 

Let  us  now  return  to  the  generic  representations  in  the  RKHS  Vg(7i),  viz.  Section  4.1. 

Thus,  we  specialize  to  elements  <j>n  =  n (xn)g.  With  Hv  =  span  {R(x)g}xeg,  let  /*  E  Hv  be 
a  signal  on  the  interval  I  of  duration  |/|  and  let  L  be  the  frame  representation  operator  for  the 
frame  {n(a;n)<7}  so  that 


LU  =  {</*,  n(xn)sr)} 


is  the  set  of  frame  coefficients. 

It  is  the  frame  coefficients  which  must  be  transmitted  or  stored.  For  representation  in  digital 
form,  it  is  necessary  that  the  frame  coefficients  be  quantized.  For  simplicity,  the  quantization 
strategy  which  we  employ  is  one  which  maps  values  uniformly  along  some  interval.  This  uniform 
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mapping  corresponds  to  specifying  each  coefficient  with  a  fixed  number  of  bits.  The  fixed 
number  of  bits  which  we  allocate  for  the  representation  of  each  coefficient  is  denoted  by  bc 
(bits/coef).  Let  the  quantization  level  set  {lk}th  a  sequence  of  increasing  real  numbers. 
The  quantization  function  Qbc  is  defined  in  terms  of  the  level  set  {lk)\l\  as 

4,  x  <C  4, 

h,  x  e  [4,4+i),  (5.3.1) 

f 2&C  ,  X  7^  * 

For  the  values  ( m,M )  =  (1,2)  and  bc  =  1,2,3  the  uniform  quantization  function  is  plotted  in 
Figure  5.2.  For  quantization  of  the  frame  coefficients  the  dynamic  range  of  interest  is  (m,  M) 
and  where 

M  =  HLAIioo  =  sup  {|  (/*,</>„)  |}  (5.3.2) 

n 

and  m  may  be  chosen  in  a  number  of  ways,  e.g.,  viz.  Section  6.3.2.  A  uniform  quantization  is 
performed  by  the  function  Qbc  if  the  level  set  {lk]\=\  has  elements 

4  =  m+  (f^r)  ( M~m )• 

We  shall  specify  the  inherent  constraint  on  the  amount  of  information  which  we  can  transmit 
per  unit  time  as  a  maximum  allowable  bit  rate  of  br  bps  (bits  per  second).  For  convenience,  we 
do  not  fix  this  quantity  explicitly.  Instead,  we  specify  a  corresponding  coefficient  rate  cr,  and 
vary  the  bit  allocation  bc  to  meet  the  information  rate  constraint  br,  through  the  simple  relation 
br  —  crbc.  With  the  coefficient  rate  fixed  and  specified,  the  maximum  number  of  coefficients  nc 
that  we  are  able  to  transmit  for  the  function  /*  of  duration  |/|  is 

nc  =  cr\I\. 

Thus,  given  the  acoustic  signal  /*  of  duration  |/|  and  a  fixed  coefficient  rate,  the  maximum 
number  of  coefficients  with  which  /«  may  be  represented,  while  still  satisfying  the  information 
rate  constraint,  is  given  by  nc.  With  respect  to  the  frame  coefficients,  this  maximum  number 
of  coefficients  nc  can  further  be  related  to  a  value  for  a  threshold  6.  To  see  this,  we  introduce 
the  coefficient  distribution  function , 

\(6)  =  dim  FfttsL(H)  =  card  {(/*,  V’m.n)  >  »  (5.3.3) 

for  6  6  [-M,  M].  The  coefficient  distribution  function  A  :  [— M,  M]  i — *  IN  is  monotonically 
decreasing  and  continuous  from  the  left.  We  may  associate  with  A  an  ‘inverse’  A-1  defined  as 

A_1(n)  =  inf  {x  E  [-M,  M]  :  A(ar)  <  n}  , 

where  n  G  IN.  If  a  threshold  value  <5  is  chosen  as 

6  =  A-1(nc), 

then  the  thresholded  frame  representation  will  have  a  cardinality 

dimT/,>A-i{nc)T(7f)  <  nc. 


Qbffx)  =  { 
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Figure  5.2:  Uniform  quantization  with  ( m,M )  =  (1,2)  for  bc  =  1,2,3. 

Consequently,  the  total  bit  requirement  for  representing  the  acoustic  signal  /»  of  duration  |/| 
in  bc  bits /coefficient  is  no  greater  than  ncbc  bits.  This,  in  turn,  guarantees  that  the  local  frame 
encoding  of  the  signal  /*  is  compatible  with  the  bit  rate  constraint,  i.e., 

bc  dim Ff,t\-i(nc)L(H)  <  br\I\. 

A  numerical  experiment  based  on  this  scheme  for  data  compression  is  detailed  in  Section 
6.3.2.  There  the  method  is  applied  to  real  speech  signals  as  well  as  synthesized  data. 
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Chapter  6 

Applications  and  Results 


In  this  chapter  we  numerically  validate  and  demonstrate  the  utility  of  the  discrete  repre¬ 
sentation  theory  developed  in  this  thesis.  For  several  test  signals  discrete  representations  are 
computed  and  displayed  for  the  cases  of  Paley- Wiener  and  wavelet  representations.  Further, 
for  the  cases  of  the  Paley- Wiener  and  wavelet  representations  the  test  signals  are  reconstructed 
from  a  variety  of  discrete  representations.  Finally,  numerical  experiments  investigating  the  noise 
suppression  and  compression  abilities  of  discrete  positive  extrema  (PE)  wavelet  representations 
are  presented. 


6.1  Test  Signals 


To  illustrate  various  discretizations  and  signal  processing  tasks,  we  have  chosen  a  set  of  four 
basic  test  signals.  Three  of  these  four  signals  are  synthetic  and  designed  to  have  interesting 
time-frequency  behavior.  The  third  signal  is  a  real  speech  signal  taken  from  the  TIMIT  1 
speech  corpus.  These  four  signals  are  respectively  labeled  “chirp”,  “packet”,  “harmonic”  and 
“water”.  Throughout  this  section  the  function  A  is  a  smooth  positive  window  function  of 
compact  support  which  is  realized  as  one  half  period  of  a  sine  wave  centered  about  the  origin, 


i.e. 


AlW  = 


sin  (£(f  +  L/2)) ,  te(-L/2,L/2) 
0,  otherwise, 


(6.1.1) 


where  L  is  the  length  of  the  interval  of  support  of  Al.  For  convenience  we  define  a  second 
window  function  A £  which  is  a  shifted  version  of  Al  having  support  starting  at  zero  as 


At  =  tl/2AL- 

We  now  describe  the  test  signals. 


Sine  Packet 

Shown  in  Figure  6.1,  is  a  synthetic  signal  which  is  a  time  progression  of  three  windowed 
sine  waves.  We  shall  refer  to  this  signal  as  “packet”.  Each  successive  windowed  sine  has  a 

’The  TIMIT  speech  corpus  is  a  data  base  consisting  of  phonemically  balanced  sentences  spoken  by  people  of 
different  dialects  throughout  the  United  States. 
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Figure  6.1:  The  signal  “packet”  and  its  magnitude  Fourier  transform. 

higher  frequency  than  then  its  predecessor.  The  windows  are  non-overlapping  and  compactly 
supported  with  an  interval  of  support  equal  to  5  periods  of  the  corresponding  sine  wave.  The 
sine  packet  sequence  is  given  by 

N 

X  -  tk)  sin(27T u>kt), 

k= l 

where  IV  =  3,  Ab/U)k  is  the  window  function  of  (6.1.1)  with  L  =  h/uk  and  (uq, 012,^3)  = 
(100,400,900) Hz. 

Chirp 

In  general,  a  chirp  signal  may  be  given  by 

Ai,{t )  sir i(af“), 

where  a  >  0  and  a  >  1  are  constants  and  Al  is  the  window  function  with  support  on  an 
interval  of  length  L.  Intuitively,  the  “instantaneous”  frequency  of  a  chirp  signal  as  a  function 
of  time  is  increasing.  The  specific  chirp  signal  which  we  use  and  refer  to  as  “chirp”  has  the 
values  a  =  0.03,  a  =  1.8.  The  signal  “chirp”  is  plotted  in  Figure  6.2. 

Harmonic 

Figure  6.3  displays  the  second  synthetic  signal  which  we  shall  refer  to  as  “harmonic”.  It 
consists  of  a  windowed  portion  of  the  superposition  of  three  sine  waves  with  harmonically  related 
frequencies.  The  harmonic  signal  is  given  by 

N 

A(t)  X  ak  sin(27rfcu>ot) 

k=i 

where  N  =  4,  A  is  the  window  function,  ak  =  1,  for  k  =  1,2,4,  03  =  0  and  lo0  =  100  Hz. 
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Figure  6.2:  Chirp  signal  and  its  magnitude  Fourier  transform. 

Female  spoken  “water” 

The  third  signal  is  a  real  speech  signal  as  spoken  by  a  female  speaker  from  the  TIMIT  data 
base.  The  word  “water”  is  taken  from  the  TIMIT  sentence  2  “She  had  your  dark  suit  in  greasy 
wash  water  all  year”.  This  signal  is  depicted  in  Figure  6.4. 

2The  exact  reference  to  the  TIMIT  data  base  is  sentence  “sal”,  speaker  “fdawO”. 
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Figure  6.3:  The  signal  “harmonic”  and  its  magnitude  Fourier  transform. 
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Figure  6.4:  Female  spoken  “water”  and  its  magnitude  Fourier  Transform. 


6.2  Numerical  Reconstruction 


The  main  question  put  to  test  the  validity  of  any  discrete  representation  is  whether  it 
contains  enough  information  about  the  underlying  signal  to  be  able  to  recover  it.  One  goal  of 
this  section  is  to  numerically  illustrate  the  theory  of  discrete  (local)  representations  developed 
in  this  thesis  on  real  signals.  In  particular,  we  shall  provide  numerous  examples  of  discrete 
representations  and  their  corresponding  reconstructions. 

We  first  examine  discrete  representations  in  the  Paley  Wiener  spaces.  Here,  we  provide 
examples  of  discrete  representations  of  bandlimited  signals  directly  by  its  samples.  Second,  we 
provide  discrete  wavelet  representations  of  the  test  signals  and  illustrate  by  example  how  such 
time-frequency  discrete  representations  are  useful  in  speech  processing  for  noise  suppression 
and  compression. 

6.2.1  Sampling  in  Paley-Wiener  Spaces 

The  most  natural  discrete  representation  of  a  signal  is  through  its  direct  sampling.  For 
bandlimited  signals  there  are  direct  sampling  discrete  representations  which  admit  full  recon¬ 
struction  of  the  original  signal,  viz.  Section  4.2.1.  In  this  section  we  provide  some  examples 
of  discrete  representations  in  bandlimited  spaces  and  numerically  illustrate  the  iterative  recon¬ 
struction  process,  viz.  Algorithms  3.4.3  and  3.5.3. 

Let  ft  >  0  be  the  finite  constant  bandlimit.  The  representation  of  a  signal  /  G  PWq  is 

Lf  =  {f(tn)},  (6-2.1) 

where  {tn}  C  ]R  is  the  sampling  set  and  L  is  the  representation  operator,  cf.  Section  4.1 
and  Table  4.2.  For  the  Paley-Wiener  space  PWn  we  examine  four  different  types  of  sampling: 


82 


uniform,  Kadec-Levinson,  jitter,  and  extrema.  Each  type  of  sampling  is  fully  explained  in 
the  sections  below.  Except  for  the  last  type  of  sampling,  all  of  these  strategies  are  signal 
independent,  i.e.  global. 

6. 2. 1.1  Uniform  Sampling 


Uniformly  sampled  representations  of  bandlimited  signals  is  the  most  commonly  employed 
representation  for  signal  processing.  The  uniformly  sampled  representation  of  a  signal  /  6  PWq 
is  given  by  (6.2.1)  where  the  sampling  set  T  is 

r  =  Tt  =  {nT} 


and  where  T  is  the  sampling  period.  For  perfect  reconstruction  of  every  signal  in  PWq  from  its 
uniform  samples  on  Ty  the  classical  sampling  Theorem  4.2.3  gives  the  necessary  and  sufficient 
condition  that  2TQ  <  1. 

Figures  6.5  through  6.8  illustrate  the  iterative  reconstruction  of  the  signals  in  the  test  set 
from  their  respective  uniform  samples.  Each  figure  indicates  the  value  of  Q,  used  to  determine 
the  sampling  period  T.  In  each  case  the  sampling  period  is  taken  to  be  T  =  1/2 Cl. 
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%  error  vs.  iteration 


Figure  6.5:  Reconstruction  of  “packet”  from  its  uniform  samples. 

The  above  figure  shows  the  uniform  representation  of  the  signal  “packet”  and  its  iterative 
reconstruction  using  Algorithm  3.4.3.  The  T2-error  is  plotted  as  a  function  of  the  iteration 
number  in  the  upper  left  graph  which  also  contains  the  values  of  the  simulation  parameters. 
To  its  right  is  the  reconstructing  dirichlet  function.  The  middle  graph  shows  the  original  signal 
and  its  uniform  samples  and  the  iterative  reconstruction  is  shown  in  the  very  bottom  graph. 
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Figure  6.6:  Reconstruction  of  “chirp”  from  its  uniform  samples. 

The  above  figure  shows  the  uniform  representation  of  the  signal  “chirp”  and  its  iterative 
reconstruction  using  Algorithm  3.4.3.  The  X2-error  is  plotted  as  a  function  of  the  iteration 
number  in  the  upper  left  graph  which  also  contains  the  values  of  the  simulation  parameters. 
To  its  right  is  the  reconstructing  dirichlet  function.  The  middle  graph  shows  the  original  signal 
and  its  uniform  samples  and  the  iterative  reconstruction  is  shown  in  the  very  bottom  graph. 
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Figure  6.7:  Reconstruction  of  “harmonic”  from  its  uniform  samples. 


The  above  figure  shows  the  uniform  representation  of  the  signal  “harmonic”  and  its  iterative 
reconstruction  using  Algorithm  3.4.3.  The  T2-error  is  plotted  as  a  function  of  the  iteration 
number  in  the  upper  left  graph  which  also  contains  the  values  of  the  simulation  parameters. 
To  its  right  is  the  reconstructing  dirichlet  function.  The  middle  graph  shows  the  original  signal 
and  its  uniform  samples  and  the  iterative  reconstruction  is  shown  in  the  very  bottom  graph. 
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Figure  6.8:  Reconstruction  of  “water”  from  its  uniform  samples. 


The  above  figure  shows  the  uniform  representation  of  the  signal  “water”  and  its  iterative 
reconstruction  using  Algorithm  3.4.3.  The  i2-error  is  plotted  as  a  function  of  the  iteration 
number  in  the  upper  left  graph  which  also  contains  the  values  of  the  simulation  parameters. 
To  its  right  is  the  reconstructing  dirichlet  function.  The  middle  graph  shows  the  original  signal 
and  its  uniform  samples  and  the  iterative  reconstruction  is  shown  in  the  very  bottom  graph. 
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6. 2. 1.2  Kadec-Levinson  Sampling 


A  Kadec-Levinson  representation  of  a  signal  /  e  PWq  is  given  by  (6.2.1)  where  the  sampling 
set  T  =  {tn}  satisfies 


for  all  n.  viz.  Definition  4.2.6.  By  Theorem  4.2.7  a  Kadec-Levinson  sequence  implies  that 
{e_<n}  is  an  exact  frame  for  PW&.  Hence,  we  may  reconstruct  any  signal  /  €  PWq  from  its 
Kadec-Levinson  representation  using  either  Algorithm  3.4.3  or  3.5.3. 

Figures  6.9  through  6.12  illustrate  the  iterative  reconstruction  of  the  signals  in  the  test 
set  from  their  respective  Kadec-Levinson  representations.  Each  figure  indicates  the  value  of  0 
used  to  determine  the  Kadec-Levinson  sampling.  In  each  case  the  sample  sequence  is  generated 
randomly  to  conform  to  the  Kadec-Levinson  condition  as 


t 


n  — 


l 

20 


where  {<rn}  is  a  sequence  of  independently  identically  distributed  random  variables  with  uniform 
distribution  on  (—1,1). 
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Figure  6.9:  Reconstruction  of  “packet”  from  its  Kadec-Levinson  samples. 


The  above  figure  shows  the  Kadec-Levinson  representation  of  the  signal  “packet”  and  its 
iterative  reconstruction  using  Algorithm  3.4.3.  The  L2-error  is  plotted  as  a  function  of  the 
iteration  number  in  the  upper  left  graph  which  also  contains  the  values  of  the  simulation 
parameters.  To  its  right  is  the  reconstructing  dirichlet  function.  The  middle  graph  shows  the 
original  signal  and  its  Kadec-Levinson  samples  and  the  iterative  reconstruction  is  shown  in  the 
very  bottom  graph. 
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Figure  6.10:  Reconstruction  of  “chirp”  from  its  Kadec-Levinson  samples. 


The  above  figure  shows  the  Kadec-Levinson  representation  of  the  signal  “chirp”  and  its 
iterative  reconstruction  using  Algorithm  3.4.3.  The  X2-error  is  plotted  as  a  function  of  the 
iteration  number  in  the  upper  left  graph  which  also  contains  the  values  of  the  simulation 
parameters.  To  its  right  is  the  reconstructing  dirichlet  function.  The  middle  graph  shows  the 
original  signal  and  its  Kadec-Levinson  samples  and  the  iterative  reconstruction  is  shown  in  the 
very  bottom  graph. 


90 


-20  0  20 
g  (ms) 


Figure  6.11:  Reconstruction  of  “harmonic”  from  its  Kadec-Levinson  samples. 

The  above  figure  shows  the  Kadec-Levinson  representation  of  the  signal  “harmonic”  and 
its  iterative  reconstruction  using  Algorithm  3.4.3.  The  Z2-error  is  plotted  as  a  function  of 
the  iteration  number  in  the  upper  left  graph  which  also  contains  the  values  of  the  simulation 
parameters.  To  its  right  is  the  reconstructing  dirichlet  function.  The  middle  graph  shows  the 
original  signal  and  its  Kadec-Levinson  samples  and  the  iterative  reconstruction  is  shown  in  the 
very  bottom  graph. 
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Figure  6.12:  Reconstruction  of  “water”  from  its  Kadec- Levinson  samples. 


The  above  figure  shows  the  Kadec-Levinson  representation  of  the  signal  “water”  and  its 
iterative  reconstruction  using  Algorithm  3.4.3.  The  X2-error  is  plotted  as  a  function  of  the 
iteration  number  in  the  upper  left  graph  which  also  contains  the  values  of  the  simulation 
parameters.  To  its  right  is  the  reconstructing  dirichlet  function.  The  middle  graph  shows  the 
original  signal  and  its  Kadec-Levinson  samples  and  the  iterative  reconstruction  is  shown  in  the 
very  bottom  graph. 
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6. 2. 1.3  Jitter  Sampling 


Jitter  sampling  sequences  are  similar  to  Kadec-Levinson  sequences  except  that  they  allow 
a  larger  constant  than  the  constant  1/4  required  in  the  Kadec-Levinson  condition.  However, 
it  is  still  necessary  that  the  jittered  sequence  be  a  uniformly  discrete  sequence,  viz.  Section 
2.5.2.  From  the  point  of  view  of  frame  reconstructions,  the  Duffin  and  Schaeffer  Theorem  4.2.4 
requires  that  the  uniform  density  Ar  of  the  sampling  sequence  T  satisfy  Ar  >  20.  If  this 
condition  is  satisfied  we  may  reconstruct  any  signal  /  €  PWq  from  its  jittered  representation 
using  either  Algorithm  3.4.3  or  3.5.3. 

Figures  6.13  through  6.16  illustrate  the  iterative  reconstruction  of  the  signals  in  the  test 
set  from  their  respective  jittered  representations.  Each  figure  indicates  the  value  of  0  used 
to  determine  the  jittered  sampling  sequence.  In  each  case  the  sample  sequence  is  generated 
randomly  as 

tn  =  (n  +  anp)  — 

where  { an }  is  a  sequence  of  independently  identically  distributed  random  variables  with  uniform 
distribution  on  (—1, 1)  and  /?  =  0.4. 
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Figure  6.13:  Reconstruction  of  “packet”  from  its  jittered  samples. 

The  above  figure  shows  the  jittered  representation  of  the  signal  “packet”  and  its  iterative 
reconstruction  using  Algorithm  3.4.3.  The  X2-error  is  plotted  as  a  function  of  the  iteration 
number  in  the  upper  left  graph  which  also  contains  the  values  of  the  simulation  parameters. 
To  its  right  is  the  reconstructing  dirichlet  function.  The  middle  graph  shows  the  original  signal 
and  its  jittered  samples  and  the  iterative  reconstruction  is  shown  in  the  very  bottom  graph. 
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Figure  6.14:  Reconstruction  of  “chirp”  from  its  jittered  samples. 

The  above  figure  shows  the  jittered  representation  of  the  signal  “chirp”  and  its  iterative 
reconstruction  using  Algorithm  3.4.3.  The  Z2-error  is  plotted  as  a  function  of  the  iteration 
number  in  the  upper  left  graph  which  also  contains  the  values  of  the  simulation  parameters. 
To  its  right  is  the  reconstructing  dirichlet  function.  The  middle  graph  shows  the  original  signal 
and  its  jittered  samples  and  the  iterative  reconstruction  is  shown  in  the  very  bottom  graph. 
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Figure  6.15:  Reconstruction  of  “harmonic”  from  its  jittered  samples. 


The  above  figure  shows  the  jittered  representation  of  the  signal  “harmonic”  and  its  iterative 
reconstruction  using  Algorithm  3.4.3.  The  T2-error  is  plotted  as  a  function  of  the  iteration 
number  in  the  upper  left  graph  which  also  contains  the  values  of  the  simulation  parameters. 
To  its  right  is  the  reconstructing  dirichlet  function.  The  middle  graph  shows  the  original  signal 
and  its  jittered  samples  and  the  iterative  reconstruction  is  shown  in  the  very  bottom  graph. 
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Figure  6.16:  Reconstruction  of  “water”  from  its  jittered  samples. 

The  above  figure  shows  the  jittered  representation  of  the  signal  “water”  and  its  iterative 
reconstruction  using  Algorithm  3.4.3.  The  X2-error  is  plotted  as  a  function  of  the  iteration 
number  in  the  upper  left  graph  which  also  contains  the  values  of  the  simulation  parameters. 
To  its  right  is  the  reconstructing  dirichlet  function.  The  middle  graph  shows  the  original  signal 
and  its  jittered  samples  and  the  iterative  reconstruction  is  shown  in  the  very  bottom  graph. 
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6. 2. 1.4  Extrema  Sampling 


Using  the  theory  of  local  frames  developed  in  Chapter  5,  we  argue  that  there  are  classes 
of  signals  for  which  their  local  extrema  locations  form  local  frames  of  complex  exponentials 
around  each  signal  in  the  class. 

To  see  this,  consider  the  windowed  sine  function  /*(t)  =  A(t)  sin(27rQ0t)  of  frequency  f l0  >  0. 
Also  consider  the  sampling  set 

r(/„)  =  r<°>(/*)Ur(1)(/*) 

where  rW(/*)  =  {f  :  flk\t)  =  oj  is  the  fcth  moment  sampling  set,  viz.  Section  5.1.1.  Here  we 
assume  that  A  is  a  strictly  positive  window  function  which  has  compact  support  in  frequency. 

For  this  /*  we  can  easily  determine  the  sampling  set  T  to  be  r(/*)  =  Thus,  r(/*) 

is  a  uniform  sampling  set  with  density  AT(/»)  =  4Cl0.  With  Cl  <  2flo,  it  follows  from  the 
Duffin-Schaeffer  Theorem  4.2.4  that  {e_tn}  is  a  frame  for  L2  [ — Ai,  Q]  or  equivalently  that  the 
set  {r_tnd2?ro}  is  a  frame  for  PWq.  Consequently  any  function  in  PWq  may  be  reconstructed 
from  its  sample  values  on  T(/„).  In  particular,  if  /,  were  in  PWq  for  some  Cl  <  2Cl0  then  it 
could  be  reconstructed  from  its  extrema  and  zero  crossing  sampling  set  F(/*)  as 

U  =  L*R*Lfm 


where  R  is  the  frame  correlation  and  L  is  the  frame  representation 

V/eiW0,  If  =  </(<)>, €r(/.c 

To  complete  the  argument  we  need  only  note  that  for  the  particular  signal  /* 

L'R*Lf.  =  L'R)Lxf+ 

where  L\  is  the  representation  associated  with  only  the  extrema  set  i.e.  L\f  — 

{/(0}ter(1)(/.)-  This  statement  can  most  easily  be  verified  through  the  observation  that  for  a 
general  frame  {4>n}  for  a  Hilbert  space  7i 

v/  €  W,  /  =  VR'Lf  =  L%LJ  =  J2  (f,  <t>n)  i’n 

where  {ipn}  is  the  dual  frame  with  frame  representation  L$.  Thus,  eliminating  terms  for  which 
(/,  4>n)  =  0  has  no  effect  on  the  reconstruction. 

If  the  window  function  is  such  that  /*  6  PWq  for  some  Cl  <  2Cl0  then  by  the  above 
arguments  we  may  conclude  that  {Ti^TrnJtgrWf/,)  *s  a  l°cal  frame  for  PWq  around  /*  and 
moreover  that  /*  can  be  reconstructed  from  only  its  extrema  values. 

Remark  6.2.1  Note  that  the  same  conclusion  can  be  made  if  the  sample  set  is  taken  as  only 
the  maxima  points  and  not  all  the  extrema  points.  In  the  case  of  /,(# )  =  A(t)  sin(27rfi0f)  the 
set  of  maxima  locations  together  with  the  set  of  zero  crossing  locations  will  have  a  density  of 
3fio- 
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It  is  clear  that  we  can  not  expect  arbitrary  bandlimited  functions  to  have  extrema  sampling 
sets  which  form  local  frames  containing  the  signal  itself.  This  is  particularly  true  for  signals  with 
complicated  time-frequency  behavior,  e.g.  the  chirp  or  sine  packet  signal.  On  the  other  hand, 
we  can  extend  the  previous  argument  fairly  easily  to  apply  to  signals  which  are  superpositions  of 
finitely  many  sinusoidal  signals.  See,  for  example,  Figure  6.19  in  which  the  signal  “harmonic”  is 
perfectly  reconstructed  from  its  extreme  points.  The  reason  we  expect  to  be  able  to  reconstruct 
a  finite  superposition  of  windowed  sinusoidals  is  that  the  highest  frequency  in  the  superposition 
will  dominate  the  extrema  set.  Consequently  the  extrema  plus  zero  set  should  have  a  density 
large  enough  to  form  a  frame  of  complex  exponentials  for  a  bandlimited  space  which  contains  the 
original  superposition.  Based  on  experiments  we  conjecture  that  there  are  large  classes  of  signals 
(other  than  finite  superpositions  of  sinusoids)  which  admit  such  local  extrema  representations. 

Consider  now  the  signal  “packet”  and  its  extrema  reconstruction  in  Figure  6.17.  It  is  evident 
from  the  figure  that  the  reconstruction  failed  to  replicate  the  original  signal;  however,  since  the 
“packet”  signal  is  a  time  progression  of  individual  sinusoidals  we  are  able  to  replicate  any 
particular  one  by  choosing  the  reconstruction  filter  g  appropriately.  In  this  figure  g  =  d27r( 0.5) 
to  match  the  middle  sinusoid.  From  the  figure  it  is  clear  that  the  sampling  for  the  lowest 
frequency  sinusoid  is  too  sparse  for  reconstruction  with  g  while  the  highest  frequency  sinusoid 
has  too  high  a  bandlimit  for  reconstruction  with  g ,  i.e.  it  is  not  in  PWy)by  For  the  middle 
sinusoid,  though,  we  are  able  to  reconstruct  perfectly  using  the  filter  g.  Similar  remarks  can  be 
made  for  the  “chirp”  signal  in  Figure  6.18. 

This  situation  suggests  the  notion  of  incorporating  different  analyzing  filters  over  different 
frequency  bands.  For  instance,  for  the  signal  “packet”  we  might  incorporate  three  different 
filters  to  reconcile  each  individual  sine  packet  and  subsequently  reconcile  (add)  the  three  results 
to  obtain  the  entire  original  signal.  This  observation  hints  strongly  at  the  wavelet  transform 
and  in  particular  the  discrete  extrema  or  positive  extrema  or  maxima  wavelet  representation  for 
signals  with  complex  time-frequency  behavior.  Discrete  wavelet  representations  are  numerically 
investigated  in  the  next  section.  Numerical  results  for  the  discrete  positive  extrema  wavelet 
representation  are  presented  in  Section  6. 2. 2.3. 

Figures  6.17  through  6.20  illustrate  the  iterative  reconstruction  of  the  signals  in  the  test  set 
from  their  extrema  representations. 
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Figure  6.17:  Reconstruction  of  “packet”  from  its  extrema  samples. 


The  above  figure  shows  the  extrema  representation  of  the  signal  “packet”  and  its  iterative 
reconstruction  using  Algorithm  3.4.3.  The  i2-error  is  plotted  as  a  function  of  the  iteration 
number  in  the  upper  left  graph  which  also  contains  the  values  of  the  simulation  parameters. 
To  its  right  is  the  reconstructing  dirichlet  function.  The  middle  graph  shows  the  original  signal 
and  its  extrema  samples  and  the  iterative  reconstruction  is  shown  in  the  very  bottom  graph. 
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Figure  6.18:  Reconstruction  of  “chirp”  from  its  extrema  samples. 


The  above  figure  shows  the  extrema  representation  of  the  signal  “chirp”  and  its  iterative 
reconstruction  using  Algorithm  3.4.3.  The  T2-error  is  plotted  as  a  function  of  the  iteration 
number  in  the  upper  left  graph  which  also  contains  the  values  of  the  simulation  parameters. 
To  its  right  is  the  reconstructing  dirichlet  function.  The  middle  graph  shows  the  original  signal 
and  its  extrema  samples  and  the  iterative  reconstruction  is  shown  in  the  very  bottom  graph. 
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Figure  6.19:  Reconstruction  of  “harmonic”  from  its  extrema  samples. 


The  above  figure  shows  the  extrema  representation  of  the  signal  “harmonic”  and  its  iterative 
reconstruction  using  Algorithm  3.4.3.  The  T2-error  is  plotted  as  a  function  of  the  iteration 
number  in  the  upper  left  graph  which  also  contains  the  values  of  the  simulation  parameters. 
To  its  right  is  the  reconstructing  dirichlet  function.  The  middle  graph  shows  the  original  signal 
and  its  extrema  samples  and  the  iterative  reconstruction  is  shown  in  the  very  bottom  graph. 
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Figure  6.20:  Reconstruction  of  “water”  from  its  extrema  samples. 


The  above  figure  shows  the  extrema  representation  of  the  signal  “water”  and  its  iterative 
reconstruction  using  Algorithm  3.4.3.  The  X2-error  is  plotted  as  a  function  of  the  iteration 
number  in  the  upper  left  graph  which  also  contains  the  values  of  the  simulation  parameters. 
To  its  right  is  the  reconstructing  dirichlet  function.  The  middle  graph  shows  the  original  signal 
and  its  extrema  samples  and  the  iterative  reconstruction  is  shown  in  the  very  bottom  graph. 
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6.2.2  Discrete  Wavelet  Representation 


In  contrast  to  the  orthogonal  case,  non-orthogonal  wavelet  frames  with  any  specifiable  shape 
and  support  in  frequency  are  easily  generated.  In  this  section  we  present  discrete  wavelet 
representations  of  the  signals  in  the  test  set  and  illustrate  the  frame  reconstructions  of  Chapter 
3. 

Recall  from  Section  4.2.3  that  the  discrete  wavelet  representation  of  a  signal  /  is  Lf  = 
{( fiTtm,nDSm9 )}  where  T  =  Q  Ga  and  g  is  the  analyzing  function.  Alternatively,  we 

may  view  this  representation  as  the  irregular  sampling  Wgf{tm<n,sm )  of  the  continuous  wavelet 
transform  Wgf  of  /. 

For  demonstration  purposes,  we  generate  an  analyzing  function  g  which  has  good  localization 
in  both  time  and  frequency.  More  specifically,  let  <7ideil  be  the  real  and  even  ideal  bandpass  filter 
specified  as 

ffideal  — 6,  —  a]  "F  l[a,6]) 

for  0  <  a  <  b  <  oo.  This  filter  is  clearly  well  localized  in  frequency,  however  its  decay  in  time 
follows  1  jt.  To  achieve  better  decay  in  time  we  convolve  in  frequency  the  ideal  bandpass  filter 
with  a  dirichlet  kernel  d^c  where  c  >  0  is  small  compared  to  b  —  a,  i.e.  c  <  b  —  a.  This  yields 
the  trapezoidal  analyzing  function  gUip  given  as 

Strap  1[— C,c]  *Sideal* 

The  specific  values  we  have  used  are  a  =  0.4,  b  =  0.5  and  c  =  0.05  KHz.  The  two  sampling 
strategies  which  we  examine  are  (i)  regular  with  respect  to  the  group  structure  of  Ga,  and  (ii) 
signal  dependent  positive  extrema  (PE). 
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Figure  6.21:  The  continuous  wavelet  transform  of  the  signal  “packet” 

6. 2. 2.1  Continuous  Wavelet  Transform 

We  have  seen  that  the  discrete  wavelet  representation  of  a  signal  /  comes  from  samples  of 
the  continuous  wavelet  transform  Wgf(t,  s),  where  ao  >  1  is  a  representation  parameter.  For  a 
given  sampling  structure  on  the  scale  axis  s  where  N  is  a  finite  integer,  the  wavelet 

transform  can  be  implemented  as  a  bank  of  N  linear  filters.  To  see  this  note  that 

Wgf(t,sm)  =  (/  *  DSmg)(t), 

so  that  the  wavelet  transform  is  the  response  of  a  bank  of  filters  with  impulse  responses 
{DSmd}m=i-  For  simplicity  the  simulations  presented  here  have  a  sampling  on  the  scale  axis 
s  which  is  fixed  according  to  {•sm}^_i  =  {a™}m=-N/2‘  As  an  example  of  a  “continuous” 
wavelet  transform  we  display  in  Figure  6.21  the  “continuous”  wavelet  transform  of  the  test 
signal  “packet”.  Actually  what  is  displayed  in  this  figure  is  the  “continuous”  output  of  the 
wavelet  filter  bank  {DSm9}m= l  where  N  =  64.  The  figure  displays  the  output  of  all  64  filters 
starting  with  the  the  response  of  D3lg  in  the  bottom  most  signal  up  to  the  response  of  DS6ig 
in  the  top  most.  In  this  figure  g  =  where  a  =  0.4  and  b  =  0.5  KHz.  The  input  signal  is 
plotted  at  the  very  top  of  the  figure  for  reference. 

6. 2. 2. 2  Regular  Wavelet  Representation 

In  this  section  we  present  a  sequence  of  figures  which  display  the  discrete  regular  wavelet 
representation  of  the  test  signals.  This  representation  is  derived  from  the  regular  samples  of 
the  continuous  wavelet  transform.  Signals  are  reconstructed  from  their  regular  wavelet  repre¬ 
sentation  via  Algorithm  3.4.3.  The  regular  wavelet  representation  of  a  signal  /  which  we  have 
used  is 

{wr,/K\«r*»T)} 

where  ao  and  T  are  chosen  so  that  this  set  is  a  regular  wavelet  frame. 
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Figures  6.22  through  6.33  illustrate  the  reconstruction  of  the  test  signals  directly  from  their 
regular  wavelet  representations. 

Figures  6.34  through  6.45  illustrate  the  reconstruction  of  the  test  signals  from  their  quan¬ 
tized  regular  wavelet  representations.  Here,  before  reconstruction  the  sample  coefficients  are 
quantized  according  to  bc  =  2  bits/coefficient,  i.e.  cq  =  Q 2(c)  is  the  quantized  data  where  Q & 
is  the  quantization  function  in  (5.3.1)  and  the  dynamic  range  ( m,M )  is  signal  dependent. 
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Figure  6.22:  Regular  wavelet  representation  for  “packet”. 


Shown  above  is  the  wavelet  regular  representation  and  its  reconstruction  for  the  signal 
“packet”.  The  trapezoidal  analyzing  function  g  appears  in  the  top  upper  right.  To  its  left 
are  the  functions  G  =  Yhm= l  \Dsmg\2  and  g.  The  middle  graph  displays  the  reconstruction  of 
Algorithm  3.4.3.  The  lower  most  graph  displays  the  sampling  set  F.  At  the  top  of  the  bottom 
graph  is  the  input  signal  and  to  its  lower  right  the  values  of  the  reconstruction  parameters. 
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Figure  6.23:  Reconstruction  of  “packet”. 


Above,  the  signal  “packet” and  its  reconstructed  versions  in  both  time  and  frequency  are 
displayed.  Below,  the  associated  coefficient  distribution  function  Apacket  is  plotted  in  the  top 
graph  while  the  lower  graph  shows  the  max  and  min  value  of  the  wavelet  filter  bank  response 
as  a  function  of  the  channel  number. 
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Figure  6.24:  Cdf  and  minimax  curves  for  “packet”. 
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Figure  6.25:  Regular  wavelet  representation  for  “chirp”. 

Shown  above  is  the  wavelet  regular  representation  and  its  reconstruction  for  the  signal 
“chirp”.  The  trapezoidal  analyzing  function  g  appears  in  the  top  upper  right.  To  its  left 
are  the  functions  G  =  \DSmg\2  and  g.  The  middle  graph  displays  the  reconstruction  of 

Algorithm  3.4.3.  The  lower  most  graph  displays  the  sampling  set  T.  At  the  top  of  the  bottom 
graph  is  the  input  signal  and  to  its  lower  right  the  values  of  the  reconstruction  parameters. 
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Figure  6.26:  Reconstruction  of  “chirp”. 

Above,  the  signal  “chirp”and  its  reconstructed  versions  in  both  time  and  frequency  are 
displayed.  Below,  the  associated  coefficient  distribution  function  Achirp  is  plotted  in  the  top 
graph  while  the  lower  graph  shows  the  max  and  min  value  of  the  wavelet  filter  bank  response 
as  a  function  of  the  channel  number. 


Figure  6.27:  Cdf  and  mini  max  curves  for  “chirp”. 


110 


0  50  100 


Reconstruction  vs.  Time  (ms) 


0  50  100 


Time(ms) 

Figure  6.28:  Regular  wavelet  representation  for  “harmonic”. 


Shown  above  is  the  wavelet  regular  representation  and  its  reconstruction  for  the  signal 
“harmonic”.  The  trapezoidal  analyzing  function  g  appears  in  the  top  upper  right.  To  its  left 
are  the  functions  G  =  J2m=l  and  g.  The  middle  graph  displays  the  reconstruction  of 

Algorithm  3.4.3.  The  lower  most  graph  displays  the  sampling  set  I\  At  the  top  of  the  bottom 
graph  is  the  input  signal  and  to  its  lower  right  the  values  of  the  reconstruction  parameters. 
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Figure  6.29:  Reconstruction  of  “harmonic”. 


Above,  the  signal  “harmonic”  and  its  reconstructed  versions  in  both  time  and  frequency  are 
displayed.  Below,  the  associated  coefficient  distribution  function  Aharmonic  is  plotted  in  the  top 
graph  while  the  lower  graph  shows  the  max  and  min  value  of  the  wavelet  filter  bank  response 
as  a  function  of  the  channel  number. 
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Figure  6.30:  Cdf  and  minimax  curves  for  “harmonic”. 
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Figure  6.31:  Regular  wavelet  representation  for  “water”. 

Shown  above  is  the  wavelet  regular  representation  and  its  reconstruction  for  the  signal 
“water”.  The  trapezoidal  analyzing  function  g  appears  in  the  top  upper  right.  To  its  left 
are  the  functions  G  =  ]Cm=i  \^sm9\2  and  9-  The  middle  graph  displays  the  reconstruction  of 
Algorithm  3.4.3.  The  lower  most  graph  displays  the  sampling  set  I\  At  the  top  of  the  bottom 
graph  is  the  input  signal  and  to  its  lower  right  the  values  of  the  reconstruction  parameters. 
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Figure  6.32:  Reconstruction  of  “water”. 


Above,  the  signal  “water”  and  its  reconstructed  versions  in  both  time  and  frequency  are 
displayed.  Below,  the  associated  coefficient  distribution  function  AWiter  is  plotted  in  the  top 
graph  while  the  lower  graph  shows  the  max  and  min  value  of  the  wavelet  filter  bank  response 
as  a  function  of  the  channel  number. 
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Figure  6.33:  Cdf  and  minimax  curves  for  “water”. 
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Figure  6.34:  Regular  wavelet  representation  for  “packet”. 


Shown  above  is  the  wavelet  regular  representation  and  its  reconstruction  for  the  signal 
“packet”.  The  trapezoidal  analyzing  function  g  appears  in  the  top  upper  right.  To  its  left 
are  the  functions  G  —  Y^m= 1  lA>m?|2  an^  9 ■  The  middle  graph  displays  the  reconstruction  of 
Algorithm  3.4.3.  The  lower  most  graph  displays  the  sampling  set  F.  At  the  top  of  the  bottom 
graph  is  the  input  signal  and  to  its  lower  right  the  values  of  the  reconstruction  parameters. 


115 


packet  (ms) 


15 

E-1 

I — T" 

T|  II  1  |  1  1  1  |  1  1 

~ 1— C 

10 

E- 

-E 

5 

L 

_E 

o 

ij 

Jta- 

T-l111  ,  ■  1  1  l  l  l  1  |  | 

,  i : 

0 

2  4  6 

B 

|packet.ft|  (KHz) 

1.5 

□ 

T  |  i  l  l  |  1  I  I  |  1 

1  L 

1 

0.5 

: 

E 

0 

ij 

JW- 

....  I-,— r-,-.|  ......  .  ...  1  i  — r 

,  i : 

0 

2  4  6 

8 

Modulus  Frequency  Reconstruction 


Figure  6.35:  Reconstruction  of  “packet”. 


Above,  the  signal  “packet” and  its  reconstructed  versions  in  both  time  and  frequency  are 
displayed.  Below,  the  associated  coefficient  distribution  function  Apicket  is  plotted  in  the  top 
graph  while  the  lower  graph  shows  the  max  and  min  value  of  the  wavelet  filter  bank  response 
as  a  function  of  the  channel  number. 
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Figure  6.36:  Cdf  and  minimax  curves  for  “packet”. 
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Figure  6.37:  Regular  wavelet  representation  for  “chirp”. 

Shown  above  is  the  wavelet  regular  representation  and  its  reconstruction  for  the  signal 
“chirp”.  The  trapezoidal  analyzing  function  g  appears  in  the  top  upper  right.  To  its  left 
are  the  functions  G  =  J2m= l  \Dsmd\2  and  g.  The  middle  graph  displays  the  reconstruction  of 
Algorithm  3.4.3.  The  lower  most  graph  displays  the  sampling  set  T.  At  the  top  of  the  bottom 
graph  is  the  input  signal  and  to  its  lower  right  the  values  of  the  reconstruction  parameters. 
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Figure  6.38:  Reconstruction  of  “chirp”. 


Above,  the  signal  “chirp”  and  its  reconstructed  versions  in  both  time  and  frequency  are 
displayed.  Below,  the  associated  coefficient  distribution  function  Achirp  is  plotted  in  the  top 
graph  while  the  lower  graph  shows  the  max  and  min  value  of  the  wavelet  filter  bank  response 
as  a  function  of  the  channel  number. 


0.2 


0.4 

cdf 


0.6 


0.8 


Figure  6.39:  Cdf  and  minimax  curves  for  “chirp”. 
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Figure  6.40:  Regular  wavelet  representation  for  “harmonic”. 


Shown  above  is  the  wavelet  regular  representation  and  its  reconstruction  for  the  signal 
“harmonic”.  The  trapezoidal  analyzing  function  g  appears  in  the  top  upper  right.  To  its  left 
are  the  functions  G  =  Ylm= 1  \Dam9\2  and  g.  The  middle  graph  displays  the  reconstruction  of 
Algorithm  3.4.3.  The  lower  most  graph  displays  the  sampling  set  T.  At  the  top  of  the  bottom 
graph  is  the  input  signal  and  to  its  lower  right  the  values  of  the  reconstruction  parameters. 
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Figure  6.41:  Reconstruction  of  “harmonic”. 


Above,  the  signal  “harmonic” and  its  reconstructed  versions  in  both  time  and  frequency  are 
displayed.  Below,  the  associated  coefficient  distribution  function  Ah„monic  is  plotted  in  the  top 
graph  while  the  lower  graph  shows  the  max  and  min  value  of  the  wavelet  filter  bank  response 
as  a  function  of  the  channel  number. 


Figure  6.42:  Cdf  and  minimax  curves  for  “harmonic”. 
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Figure  6.43:  Regular  wavelet  representation  for  “water”. 


Shown  above  is  the  wavelet  regular  representation  and  its  reconstruction  for  the  signal 
“water”.  The  trapezoidal  analyzing  function  g  appears  in  the  top  upper  right.  To  its  left 
are  the  functions  G  =  J2m= 1  and  g.  The  middle  graph  displays  the  reconstruction  of 

Algorithm  3.4.3.  The  lower  most  graph  displays  the  sampling  set  I\  At  the  top  of  the  bottom 
graph  is  the  input  signal  and  to  its  lower  right  the  values  of  the  reconstruction  parameters. 
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Figure  6.44:  Reconstruction  of  “water”. 


Above,  the  signal  “water”  and  its  reconstructed  versions  in  both  time  and  frequency  are 
displayed.  Below,  the  associated  coefficient  distribution  function  Awater  is  plotted  in  the  top 
graph  while  the  lower  graph  shows  the  max  and  min  value  of  the  wavelet  filter  bank  response 
as  a  function  of  the  channel  number. 
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Figure  6.45:  Cdf  and  minimax  curves  for  “water”. 
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6. 2. 2. 3  Positive  Extrema  Wavelet  Representation 

In  this  section  we  present  a  sequence  of  figures  which  display  the  discrete  positive  extrema 
wavelet  representation  of  the  test  signals.  This  representation  is  derived  from  the  extrema 
samples  of  the  continuous  wavelet  transform  which  are  positive.  Signals  are  reconstructed 
from  their  positive  extrema  (PE)  wavelet  representation  via  Algorithm  3.4.3.  The  PE  wavelet 
representation  of  a  signal  /  which  we  have  used  is 

where 

=  :  wgf(ag,t)>  0,  dtWgf(a%,t)  =  0}  . 

Figures  6.46  through  6.57  illustrate  the  reconstruction  of  the  signals  in  the  test  set  from 
their  positive  extrema  wavelet  representations. 
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Figure  6.46:  Positive  extrema  wavelet  representation  for  “packet”. 


Shown  above  is  the  wavelet  positive  extrema  representation  and  its  reconstruction  for  the 
signal  “packet”.  The  trapezoidal  analyzing  function  g  appears  in  the  top  upper  right.  To  its  left 
are  the  functions  G  =  \^sms\2  and  <?•  The  middle  graph  displays  the  reconstruction  of 

Algorithm  3.4.3.  The  lower  most  graph  displays  the  sampling  set  T.  At  the  top  of  the  bottom 
graph  is  the  input  signal  and  to  its  lower  right  the  values  of  the  reconstruction  parameters. 


124 


|paeket.ft|  (KHz) 


Modulus  Frequency  Reconstruction 


Figure  6.47:  Reconstruction  of  “packet”. 


Above,  the  signal  “packet”and  its  reconstructed  versions  in  both  time  and  frequency  are 
displayed.  Below,  the  associated  coefficient  distribution  function  Apacket  is  plotted  in  the  top 
graph  while  the  lower  graph  shows  the  max  and  min  value  of  the  wavelet  filter  bank  response 
as  a  function  of  the  channel  number. 


Figure  6.48:  Cdf  and  minimax  curves  for  “packet”. 
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Figure  6.49:  Positive  extrema  wavelet  representation  for  “chirp”. 


Shown  above  is  the  wavelet  positive  extrema  representation  and  its  reconstruction  for  the 
signal  “chirp”.  The  trapezoidal  analyzing  function  g  appears  in  the  top  upper  right.  To  its  left 
are  the  functions  G  =  Ylm-i  \DSmg\2  and  g.  The  middle  graph  displays  the  reconstruction  of 
Algorithm  3.4.3.  The  lower  most  graph  displays  the  sampling  set  T.  At  the  top  of  the  bottom 
graph  is  the  input  signal  and  to  its  lower  right  the  values  of  the  reconstruction  parameters. 
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Figure  6.50:  Reconstruction  of  “chirp”. 


Above,  the  signal  “chirp”  and  its  reconstructed  versions  in  both  time  and  frequency  are 
displayed.  Below,  the  associated  coefficient  distribution  function  Achirp  is  plotted  in  the  top 
graph  while  the  lower  graph  shows  the  max  and  min  value  of  the  wavelet  filter  bank  response 
as  a  function  of  the  channel  number. 
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Figure  6.51:  Cdf  and  minimax  curves  for  “chirp”. 
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Figure  6.52:  Positive  extrema  wavelet  representation  for  “harmonic”. 


Shown  above  is  the  wavelet  positive  extrema  representation  and  its  reconstruction  for  the 
signal  “harmonic”.  The  trapezoidal  analyzing  function  g  appears  in  the  top  upper  right.  To  its 
left  are  the  functions  G  =  £3m=i  \DSmg\2  and  g.  The  middle  graph  displays  the  reconstruction 
of  Algorithm  3.4.3.  The  lower  most  graph  displays  the  sampling  set  T.  At  the  top  of  the  bottom 
graph  is  the  input  signal  and  to  its  lower  right  the  values  of  the  reconstruction  parameters. 
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Figure  6.53:  Reconstruction  of  “harmonic”. 


Above,  the  signal  “harmonic”  and  its  reconstructed  versions  in  both  time  and  frequency  are 
displayed.  Below,  the  associated  coefficient  distribution  function  Aharrnonic  is  plotted  in  the  top 
graph  while  the  lower  graph  shows  the  max  and  min  value  of  the  wavelet  filter  bank  response 
as  a  function  of  the  channel  number. 


Figure  6.54:  Cdf  and  minimax  curves  for  “harmonic”. 
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Figure  6.55:  Positive  extrema  wavelet  representation  for  “water”. 


Shown  above  is  the  wavelet  positive  extrema  representation  and  its  reconstruction  for  the 
signal  “water”.  The  trapezoidal  analyzing  function  g  appears  in  the  top  upper  right.  To  its  left 
are  the  functions  G  =  J2m= i  I DSjng\2  and  g.  The  middle  graph  displays  the  reconstruction  of 
Algorithm  3.4.3.  The  lower  most  graph  displays  the  sampling  set  T.  At  the  top  of  the  bottom 
graph  is  the  input  signal  and  to  its  lower  right  the  values  of  the  reconstruction  parameters. 
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Figure  6.56:  Reconstruction  of  “water”. 


Above,  the  signal  “water” and  its  reconstructed  versions  in  both  time  and  frequency  are 
displayed.  Below,  the  associated  coefficient  distribution  function  Awi,er  is  plotted  in  the  top 
graph  while  the  lower  graph  shows  the  max  and  min  value  of  the  wavelet  filter  bank  response 
as  a  function  of  the  channel  number. 
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Figure  6.57:  Cdf  and  minimax  curves  for  “water”. 
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6.3  Applications 

We  have  seen  numerous  examples  of  discrete  wavelet  representations  in  the  previous  sections. 
From  these  examples  it  is  seen  that  these  discrete  wavelet  representations  characterize  signals 
in  a  manner  which  reflects  their  time-frequency  behavior.  A  representation  of  this  sort  lends 
itself  to  numerous  applications.  Chapter  7  presents  a  partial  list  of  such  applications.  In  this 
section  we  limit  ourselves  to  two  applications:  noise  suppression  and  compression. 

For  the  discrete  positive  extrema  wavelet  representation  with  the  trapezoidal  analyzing 
function  <7,rap,  the  next  two  sections  display  numerical  results  of  (i)  the  noise  suppression  scheme 
discussed  in  Section  3.5.2  and  (ii)  the  compression  scheme  discussed  in  Section  5.3. 

6.3.1  Noise  Suppression 

For  the  noise  suppression  problem  the  goal  is  to  reject  noisy  portions  of  corrupted  signals 
and  to  recover  their  coherent  portions.  Based  on  the  scheme  discussed  in  Section  3.5.2,  a 
numerical  experiment  is  presented  and  its  performance  in  various  levels  of  random  additive 
noise  is  displayed. 

The  noise  suppression  experiment  can  be  summarized  as  follows.  Fix  a  noise  level  a  >  0 
and  a  threshold  6  >  0.  For  each  /*  in  the  test  set  the  following  steps  are  performed: 

(i)  Generate  a  random  noise  signal  w(t )  such  that  for  each  t,  w(t )  is  uniformly  distributed 
in  [-1,1]. 

(ii)  Additively  corrupt  the  signal  /*  by  aw  yielding  the  noisy  signal 

fw  =  S*  +  °w- 

(iii)  Compute  the  wavelet  positive  extrema  representation  of  fw: 

MU)  =  {Wgfw(tm,n,Sm)}  ■ 

(iv)  Compute  the  distribution  function  A fw. 

(v)  Threshold  the  wavelet  positive  extrema  representation  A (/*)  by  6,  yielding  the  truncated 
representation 

A s(U)  =  Sm)  •  Ifp  fw{tm,nt  Sm  )>*}■ 

(vi)  Generate  reconstruction  of  /*  using  Algorithm  3.4.3  where  initial  data  is 


co  —  A.s(fw)- 


Figures  6.58  through  6.69  display  the  noise  suppressed  reconstructions  for  an  additive  noise 
level  a  =  0.3  corresponding  to  a  signal  to  noise  ratio  of  SNR  «  5.2dB. 

Figures  6.70  through  6.81  display  the  noise  suppressed  reconstructions  for  an  additive  noise 
level  a  —  0.5  corresponding  to  a  signal  to  noise  ratio  of  SNR  «  3.0dB. 

Figures  6.82  through  6.93  display  the  noise  suppressed  reconstructions  for  an  additive  noise 
level  a  —  0.7  corresponding  to  a  signal  to  noise  ratio  of  SNR  k  1.5dB. 
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Figure  6.58:  PE  wavelet  representation  for  “packet”,  a  =  0.3. 


Shown  above  is  the  wavelet  positive  extrema  representation  and  its  reconstruction  for  the 
noisy  signal  “packet”.  The  trapezoidal  analyzing  function  g  appears  in  the  top  upper  right. 
To  its  left  are  the  functions  G  =  \DSmg\2  and  g.  The  middle  graph  displays  the  recon¬ 

struction  of  Algorithm  3.4.3.  The  lower  most  graph  displays  the  sampling  set  T.  At  the  top 
of  the  bottom  graph  is  the  input  signal  and  to  its  lower  right  the  values  of  the  reconstruction 
parameters. 
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Figure  6.59:  Reconstruction  of  noisy  “packet”,  a  —  0.3. 


Above,  the  signal  “packet”  and  its  reconstructed  versions  in  both  time  and  frequency  are 
displayed.  Below,  the  associated  coefficient  distribution  function  Apacl[ct  is  plotted  in  the  top 
graph  while  the  lower  graph  shows  the  max  and  min  value  of  the  wavelet  filter  bank  response 
as  a  function  of  the  channel  number. 


Figure  6.60:  Cdf  and  minimax  curves  for  noisy  “packet”,  a  —  0.3. 
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Figure  6.61:  PE  wavelet  representation  for  “chirp”,  a  —  0.3. 

Shown  above  is  the  wavelet  positive  extrema  representation  and  its  reconstruction  for  the 
noisy  signal  “chirp”.  The  trapezoidal  analyzing  function  g  appears  in  the  top  upper  right.  To  its 
left  are  the  functions  G  =  J2m-i  \^smg\2  and  g.  The  middle  graph  displays  the  reconstruction 
of  Algorithm  3.4.3.  The  lower  most  graph  displays  the  sampling  set  T.  At  the  top  of  the  bottom 
graph  is  the  input  signal  and  to  its  lower  right  the  values  of  the  reconstruction  parameters.  , 
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Figure  6.62:  Reconstruction  of  noisy  “chirp”,  cr  =  0.3. 


Above,  the  signal  “chirp”  and  its  reconstructed  versions  in  both  time  and  frequency  are 
displayed.  Below,  the  associated  coefficient  distribution  function  Achirp  is  plotted  in  the  top 
graph  while  the  lower  graph  shows  the  max  and  min  value  of  the  wavelet  filter  bank  response 
as  a  function  of  the  channel  number. 


Figure  6.63:  Cdf  and  minimax  curves  for  noisy  “chirp”,  <j  =  0.3. 
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Shown  above  is  the  wavelet  positive  extrema  representation  and  its  reconstruction  for  the 
noisy  signal  “harmonic”.  The  trapezoidal  analyzing  function  g  appears  in  the  top  upper  right. 
To  its  left  are  the  functions  G  =  J2m=i  l-^sm?|2  and  g.  The  middle  graph  displays  the  recon¬ 
struction  of  Algorithm  3.4.3.  The  lower  most  graph  displays  the  sampling  set  T.  At  the  top 
of  the  bottom  graph  is  the  input  signal  and  to  its  lower  right  the  values  of  the  reconstruction 
parameters. 
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Figure  6.65:  Reconstruction  of  noisy  “harmonic”,  a  =  0.3. 

Above,  the  signal  “harmonic” and  its  reconstructed  versions  in  both  time  and  frequency  are 
displayed.  Below,  the  associated  coefficient  distribution  function  Aharmonic  is  plotted  in  the  top 
graph  while  the  lower  graph  shows  the  max  and  min  value  of  the  wavelet  filter  bank  response 
as  a  function  of  the  channel  number. 


Figure  6.66:  Cdf  and  minimax  curves  for  noisy  “harmonic”,  a  —  0.3. 
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Figure  6.67:  PE  wavelet  representation  for  “water”,  a  =  0.3. 

Shown  above  is  the  wavelet  positive  extrema  representation  and  its  reconstruction  for  the 
noisy  signal  “water”.  The  trapezoidal  analyzing  function  g  appears  in  the  top  upper  right.  To  its 
left  are  the  functions  G  =  52m=i  I Dsmd\2  and  g ■  The  middle  graph  displays  the  reconstruction 
of  Algorithm  3.4.3.  The  lower  most  graph  displays  the  sampling  set  T.  At  the  top  of  the  bottom 
graph  is  the  input  signal  and  to  its  lower  right  the  values  of  the  reconstruction  parameters. 
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Figure  6.68:  Reconstruction  of  noisy  “water”,  a  =  0.3. 


Above,  the  signal  “water”  and  its  reconstructed  versions  in  both  time  and  frequency  are 
displayed.  Below,  the  associated  coefficient  distribution  function  AWiter  is  plotted  in  the  top 
graph  while  the  lower  graph  shows  the  max  and  min  value  of  the  wavelet  filter  bank  response 
as  a  function  of  the  channel  number. 


Figure  6.69:  Cdf  and  minimax  curves  for  noisy  “water”,  a  —  0.3. 


Figure  6.70:  PE  wavelet  representation  for  “packet”,  a  —  0.5. 


Shown  above  is  the  wavelet  positive  extrema  representation  and  its  reconstruction  for  the 
noisy  signal  “packet”.  The  trapezoidal  analyzing  function  g  appears  in  the  top  upper  right. 
To  its  left  are  the  functions  G  =  YLm= 1  \Dsmg\2  and  g.  The  middle  graph  displays  the  recon¬ 
struction  of  Algorithm  3.4.3.  The  lower  most  graph  displays  the  sampling  set  T.  At  the  top 
of  the  bottom  graph  is  the  input  signal  and  to  its  lower  right  the  values  of  the  reconstruction 
parameters. 
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Figure  6.71:  Reconstruction  of  noisy  “packet”,  a  —  0.5. 


Above,  the  signal  “packet” and  its  reconstructed  versions  in  both  time  and  frequency  are 
displayed.  Below,  the  associated  coefficient  distribution  function  Apacket  is  plotted  in  the  top 
graph  while  the  lower  graph  shows  the  max  and  min  value  of  the  wavelet  filter  bank  response 
as  a  function  of  the  channel  number. 


Figure  6.72:  Cdf  and  minimax  curves  for  noisy  “packet”,  a  =  0.5. 
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Figure  6.73:  PE  wavelet  representation  for  “chirp”,  a  =  0.5. 


Shown  above  is  the  wavelet  positive  extrema  representation  and  its  reconstruction  for  the 
noisy  signal  “chirp”.  The  trapezoidal  analyzing  function  g  appears  in  the  top  upper  right.  To  its 
left  are  the  functions  G  =  J2m=i  \DSmg\2  and  g.  The  middle  graph  displays  the  reconstruction 
of  Algorithm  3.4.3.  The  lower  most  graph  displays  the  sampling  set  T.  At  the  top  of  the  bottom 
graph  is  the  input  signal  and  to  its  lower  right  the  values  of  the  reconstruction  parameters. 
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Figure  6.74:  Reconstruction  of  noisy  “chirp”,  a  —  0.5. 


Above,  the  signal  “chirp”  and  its  reconstructed  versions  in  both  time  and  frequency  are 
displayed.  Below,  the  associated  coefficient  distribution  function  Achirp  is  plotted  in  the  top 
graph  while  the  lower  graph  shows  the  max  and  min  value  of  the  wavelet  filter  bank  response 
as  a  function  of  the  channel  number. 


Figure  6.75:  Cdf  and  minimax  curves  for  noisy  “chirp”,  a  —  0.5. 
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Figure  6.76:  PE  wavelet  representation  for  “harmonic”,  cr  =  0.5. 


Shown  above  is  the  wavelet  positive  extrema  representation  and  its  reconstruction  for  the 
noisy  signal  “harmonic”.  The  trapezoidal  analyzing  function  g  appears  in  the  top  upper  right. 
To  its  left  are  the  functions  G  =  Ylm=i  \DSmg\2  and  g.  The  middle  graph  displays  the  recon¬ 
struction  of  Algorithm  3.4.3.  The  lower  most  graph  displays  the  sampling  set  T.  At  the  top 
of  the  bottom  graph  is  the  input  signal  and  to  its  lower  right  the  values  of  the  reconstruction 
parameters. 


145 


0  50  100  0  2  4  6  8 

noisy  harmonic  (ms)  |noisy  harmonic.ft|  (KHz) 


Time  Reconstruction  Modulus  Frequency  Reconstruction 

Figure  6.77:  Reconstruction  of  noisy  “harmonic”,  cr  =  0.5. 


Above,  the  signal  “harmonic” and  its  reconstructed  versions  in  both  time  and  frequency  are 
displayed.  Below,  the  associated  coefficient  distribution  function  Aharmonic  is  plotted  in  the  top 
graph  while  the  lower  graph  shows  the  max  and  min  value  of  the  wavelet  filter  bank  response 
as  a  function  of  the  channel  number. 


Figure  6.78:  Cdf  and  minimax  curves  for  noisy  “harmonic”,  a  —  0.5. 
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Figure  6.79:  PE  wavelet  representation  for  “water”,  a  =  0.5. 


Shown  above  is  the  wavelet  positive  extrema  representation  and  its  reconstruction  for  the 
noisy  signal  “water” .  The  trapezoidal  analyzing  function  g  appears  in  the  top  upper  right .  To  its 
left  are  the  functions  G  =  J2m= l  |-Dsm?|2  and  g.  The  middle  graph  displays  the  reconstruction 
of  Algorithm  3.4.3.  The  lower  most  graph  displays  the  sampling  set  T.  At  the  top  of  the  bottom 
graph  is  the  input  signal  and  to  its  lower  right  the  values  of  the  reconstruction  parameters. 
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Figure  6.80:  Reconstruction  of  noisy  “water”,  a  =  0.5. 


Above,  the  signal  “water” and  its  reconstructed  versions  in  both  time  and  frequency  are 
displayed.  Below,  the  associated  coefficient  distribution  function  AwateI  is  plotted  in  the  top 
graph  while  the  lower  graph  shows  the  max  and  min  value  of  the  wavelet  filter  bank  response 
as  a  function  of  the  channel  number. 


Figure  6.81:  Cdf  and  minimax  curves  for  noisy  “water”,  a  —  0.5. 
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Shown  above  is  the  wavelet  positive  extrema  representation  and  its  reconstruction  for  the 
noisy  signal  “packet”.  The  trapezoidal  analyzing  function  g  appears  in  the  top  upper  right. 
To  its  left  are  the  functions  G  —  J2m=i  \Dam{j\2  and  g.  The  middle  graph  displays  the  recon¬ 
struction  of  Algorithm  3.4.3.  The  lower  most  graph  displays  the  sampling  set  F.  At  the  top 
of  the  bottom  graph  is  the  input  signal  and  to  its  lower  right  the  values  of  the  reconstruction 
parameters. 
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Figure  6.83:  Reconstruction  of  noisy  “packet”,  a  —  0.7. 


Above,  the  signal  “packet”and  its  reconstructed  versions  in  both  time  and  frequency  are 
displayed.  Below,  the  associated  coefficient  distribution  function  Apicket  is  plotted  in  the  top 
graph  while  the  lower  graph  shows  the  max  and  min  value  of  the  wavelet  filter  bank  response 
as  a  function  of  the  channel  number. 
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Figure  6.84:  Cdf  and  minimax  curves  for  noisy  “packet”,  a  —  0.7. 
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Figure  6.85:  PE  wavelet  representation  for  “chirp”,  a  =  0.7. 

Shown  above  is  the  wavelet  positive  extrema  representation  and  its  reconstruction  for  the 
noisy  signal  “chirp”.  The  trapezoidal  analyzing  function  g  appears  in  the  top  upper  right.  To  its 
left  are  the  functions  G  =  J2m=i  I and  g-  The  middle  graph  displays  the  reconstruction 
of  Algorithm  3.4.3.  The  lower  most  graph  displays  the  sampling  set  T.  At  the  top  of  the  bottom 
graph  is  the  input  signal  and  to  its  lower  right  the  values  of  the  reconstruction  parameters. 
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Figure  6.86:  Reconstruction  of  noisy  “chirp”,  a  =  0.7. 

Above,  the  signal  “chirp”  and  its  reconstructed  versions  in  both  time  and  frequency  are 
displayed.  Below,  the  associated  coefficient  distribution  function  Achirp  is  plotted  in  the  top 
graph  while  the  lower  graph  shows  the  max  and  min  value  of  the  wavelet  filter  bank  response 
as  a  function  of  the  channel  number. 


Figure  6.87:  Cdf  and  minimax  curves  for  noisy  “chirp”,  a  —  0.7. 
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Figure  6.88:  PE  wavelet  representation  for  “harmonic”,  a  —  0.7. 


Shown  above  is  the  wavelet  positive  extrema  representation  and  its  reconstruction  for  the 
noisy  signal  “harmonic”.  The  trapezoidal  analyzing  function  g  appears  in  the  top  upper  right. 
To  its  left  are  the  functions  G  =  J2m= 1  \DSmg\2  and  g.  The  middle  graph  displays  the  recon¬ 
struction  of  Algorithm  3.4.3.  The  lower  most  graph  displays  the  sampling  set  T.  At  the  top 
of  the  bottom  graph  is  the  input  signal  and  to  its  lower  right  the  values  of  the  reconstruction 
parameters. 
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Figure  6.89:  Reconstruction  of  noisy  “harmonic”,  a  =  0.7. 


Above,  the  signal  “harmonic”  and  its  reconstructed  versions  in  both  time  and  frequency  are 
displayed.  Below,  the  associated  coefficient  distribution  function  Allirmonic  is  plotted  in  the  top 
graph  while  the  lower  graph  shows  the  max  and  min  value  of  the  wavelet  filter  bank  response 
as  a  function  of  the  channel  number. 
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Figure  6.90:  Cdf  and  minimax  curves  for  noisy  “harmonic”,  a  —  0.7. 
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Figure  6.91:  PE  wavelet  representation  for  “water”,  a  =  0.7. 


Shown  above  is  the  wavelet  positive  extrema  representation  and  its  reconstruction  for  the 
noisy  signal  “water”.  The  trapezoidal  analyzing  function  g  appears  in  the  top  upper  right.  To  its 
left  are  the  functions  G  =  J2m=i  and  g.  The  middle  graph  displays  the  reconstruction 

of  Algorithm  3.4.3.  The  lower  most  graph  displays  the  sampling  set  T.  At  the  top  of  the  bottom 
graph  is  the  input  signal  and  to  its  lower  right  the  values  of  the  reconstruction  parameters. 
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Figure  6.92:  Reconstruction  of  noisy  “water”,  a  =  0.7. 


Above,  the  signal  “water”  and  its  reconstructed  versions  in  both  time  and  frequency  are 
displayed.  Below,  the  associated  coefficient  distribution  function  AWiter  is  plotted  in  the  top 
graph  while  the  lower  graph  shows  the  max  and  min  value  of  the  wavelet  filter  bank  response 
as  a  function  of  the  channel  number. 


Figure  6.93:  Cdf  and  minimax  curves  for  noisy  “water”,  a  =  0.7. 
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6.3.2  Compression 

In  speech,  the  goal  of  compression  is  to  represent  speech  signals  in  a  way  which  minimizes 
storage  and  transmission  bandwidth  requirements  under  the  constraint  that  sufficiently  high 
“quality”  approximations  of  the  original  speech  signal  can  be  recovered  from  the  representation. 
The  meaning  of  the  ‘quality’  of  a  reconstruction  is  a  criterion  which  is  difficult  to  specify 
precisely.  In  vague  terms  we  would  like  our  representation  to  preserve  pertinent  perceptual 
information  in  the  speech  signal,  e.g.,  timbre,  emotional  state  of  the  speaker,  inflections,  etc. 
Intelligibility  is  a  less  stringent  criterion  by  which  to  judge  reconstructions.  In  this  case,  we 
require  only  that  listeners  be  able  to  determine  the  textual  content  of  the  original  speech  signal 
purely  from  audition  of  the  reconstruction. 

We  detail  a  compression  experiment  based  on  the  discussion  in  Section  5.3.  Let  g  =  <7trap- 
Fix  a  coefficient  rate  cr,  e.g.,  cT  =  4800  coef/sec.  For  each  /,  in  the  test  set  and  for  each  value 
of  bc ,  the  following  steps  are  performed: 

(i)  Compute  the  wavelet  positive  extrema  representation  of  /*: 

A(/*)  =  {WgMtmtn,sm)}. 

(ii)  Determine  the  maximum  number  of  coefficients  nc  with  which  /*  can  be  represented  and 
still  met  coefficient  rate  constraint: 

nc  =  cT  •  |/(/*)|, 

where  |/(/*)|  is  the  duration  of  /*. 

(iii)  Compute  the  distribution  function  A/,. 

(iv)  Threshold  the  wavelet  positive  extrema  representation  A (/*)  by  S  =  A_1(rcc),  yielding  the 
truncated  representation 

Aj(/*)  —  Sm)  .  fFy  f * (tm,n ,  ^  <5}  . 

(v)  Quantize  the  thresholded  wavelet  positive  extrema  representation,  yielding  the  sequence 

QbXHf*))- 

(vi)  Generate  reconstruction  of  /*  using  Algorithm  3.5.3  where  initial  data  is 

co  =  Qbc(hs(f*)). 


Synthetic  Data 

For  the  compression  of  the  non-speech  signals  in  the  test  set  we  have  chosen  the  discretization 
parameters  g  =  ff,rip  and  ao  =  1.09. 

Figures  6.94  through  6.102  display  the  compressed  reconstructions  for  a  bit  rate  constraint 
of  br  =  1.2  coefs/msec  and  a  quantization  of  bc  =  2  bits/coef,  i.e.,  2.4  Kbits/sec. 

Figures  6.103  through  6.111  display  the  compressed  reconstructions  for  a  bit  rate  constraint 
of  br  —  1.2  coefs/msec  and  a  quantization  of  bc  =  1  bit/coef,  i.e.,  1.2  Kbits/sec. 
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Speech  Data 

The  discretization  parameters  we  have  chosen  for  the  speech  data  are  ao  =  1.1  and  g  =  gUip. 
Note  that  to  achieve  greater  compression  we  have  resampled  the  s-axis  effectively  eliminating 
every  other  channel. 

Figures  6.112  through  6.120  display  the  compressed  reconstructions  for  a  bit  rate  constraint 
of  bT  =  9.6  coefs/msec  and  a  quantization  levels  of  bc  =  4,2  and  1  bit/coef  respectively, 
corresponding  to  bit  rates  of  38.4,  19.2,  and  9.6  Kbits/sec. 

Figures  6.121  through  6.129  display  the  compressed  reconstructions  for  a  bit  rate  constraint 
of  br  =  4.8  coefs/msec  and  a  quantization  levels  of  bc  =  4,2  and  1  bit/coef  respectively, 
corresponding  to  bit  rates  of  19.2,  9.6,  and  4.8  Kbits/sec. 

Figures  6.130  through  6.138  display  the  compressed  reconstructions  for  a  bit  rate  constraint 
of  bT  =  2.4  coefs/msec  and  a  quantization  levels  of  bc  =  4,2  and  1  bit/coef  respectively, 
corresponding  to  bit  rates  of  9.6,  4.8  and  2.4  Kbits/sec. 
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Figure  6.94:  PE  wavelet  representation  for  “packet”,  ( bc,br )  =  (2, 1.2). 

Shown  above  is  the  wavelet  positive  extrema  representation  and  its  reconstruction  for  the 
compressed  signal  “packet”.  The  trapezoidal  analyzing  function  g  appears  in  the  top  upper 
right.  To  its  left  are  the  functions  G  —  X2m=i  | DSmg\2  and  g.  The  middle  graph  displays  the 
reconstruction  of  Algorithm  3.4.3.  The  lower  most  graph  displays  the  sampling  set  F.  At  the 
top  of  the  bottom  graph  is  the  input  signal  and  to  its  lower  right  the  values  of  the  reconstruction 
parameters. 
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Figure  6.95:  Reconstruction  of  compressed  “packet”,  ( bc,br )  =  (2,1.2). 


Above,  the  signal  “packet”and  its  reconstructed  versions  in  both  time  and  frequency  are 
displayed.  Below,  the  associated  coefficient  distribution  function  Apack(!,  is  plotted  in  the  top 
graph  while  the  lower  graph  shows  the  max  and  min  value  of  the  wavelet  filter  bank  response 
as  a  function  of  the  channel  number. 
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Figure  6.96:  Cdf  and  minimax  curves  for  compressed  “packet”,  (bc,br)  =  (2, 1.2). 
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Figure  6.97:  PE  wavelet  representation  for  “chirp”,  ( bc,br )  =  (2, 1.2). 


Shown  above  is  the  wavelet  positive  extrema  representation  and  its  reconstruction  for  the 
compressed  signal  “chirp”.  The  trapezoidal  analyzing  function  g  appears  in  the  top  upper 
right.  To  its  left  are  the  functions  G  =  ]Cm=i  \^smd\2  and  g-  The  middle  graph  displays  the 
reconstruction  of  Algorithm  3.4.3.  The  lower  most  graph  displays  the  sampling  set  T.  At  the 
top  of  the  bottom  graph  is  the  input  signal  and  to  its  lower  right  the  values  of  the  reconstruction 
parameters. 
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Figure  6.98:  Reconstruction  of  compressed  “chirp”,  ( bc,br )  =  (2,1.2). 


Above,  the  signal  “chirp” and  its  reconstructed  versions  in  both  time  and  frequency  are 
displayed.  Below,  the  associated  coefficient  distribution  function  Achirp  is  plotted  in  the  top 
graph  while  the  lower  graph  shows  the  max  and  min  value  of  the  wavelet  filter  bank  response 
as  a  function  of  the  channel  number. 
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Figure  6.99:  Cdf  and  minimax  curves  for  compressed  “chirp”,  ( bc,bT )  =  (2, 1.2). 
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Figure  6.100:  PE  wavelet  representation  for  “harmonic”,  ( bc,br )  =  (2, 1.2). 


Shown  above  is  the  wavelet  positive  extrema  representation  and  its  reconstruction  for  the 
compressed  signal  “harmonic”.  The  trapezoidal  analyzing  function  g  appears  in  the  top  upper 
right.  To  its  left  are  the  functions  G  =  Ylm= l  |A>m?|2  and  g.  The  middle  graph  displays  the 
reconstruction  of  Algorithm  3.4.3.  The  lower  most  graph  displays  the  sampling  set  F.  At  the 
top  of  the  bottom  graph  is  the  input  signal  and  to  its  lower  right  the  values  of  the  reconstruction 
parameters. 
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Figure  6.101:  Reconstruction  of  compressed  “harmonic”,  ( bc,br )  =  (2,1.2). 


Above,  the  signal  “harmonic” and  its  reconstructed  versions  in  both  time  and  frequency  are 
displayed.  Below,  the  associated  coefficient  distribution  function  Aharmonic  is  plotted  in  the  top 
graph  while  the  lower  graph  shows  the  max  and  min  value  of  the  wavelet  filter  bank  response 
as  a  function  of  the  channel  number. 


Figure  6.102:  Cdf  and  minimax  curves  for  compressed  “harmonic”,  (bc,br)  =  (2, 1.2). 
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Figure  6.103:  PE  wavelet  representation  for  “packet”,  ( bc,br )  =  (1, 1.2). 


Shown  above  is  the  wavelet  positive  extrema  representation  and  its  reconstruction  for  the 
compressed  signal  “packet”.  The  trapezoidal  analyzing  function  g  appears  in  the  top  upper 
right.  To  its  left  are  the  functions  G  =  J2m=i  \Dsm9\2  and  g.  The  middle  graph  displays  the 
reconstruction  of  Algorithm  3.4.3.  The  lower  most  graph  displays  the  sampling  set  I\  At  the 
top  of  the  bottom  graph  is  the  input  signal  and  to  its  lower  right  the  values  of  the  reconstruction 
parameters. 
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Figure  6.104:  Reconstruction  of  compressed  “packet”,  (bc,bT)  =  (1,1.2). 


Above,  the  signal  “packet”and  its  reconstructed  versions  in  both  time  and  frequency  are 
displayed.  Below,  the  associated  coefficient  distribution  function  Apacket  is  plotted  in  the  top 
graph  while  the  lower  graph  shows  the  max  and  min  value  of  the  wavelet  filter  bank  response 
as  a  function  of  the  channel  number. 


Figure  6.105:  Cdf  and  minimax  curves  for  compressed  “packet”,  (bc, br)  =  (1, 1.2). 
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Figure  6.106:  PE  wavelet  representation  for  “chirp”,  ( bc,br )  =  (1, 1.2). 


Shown  above  is  the  wavelet  positive  extrema  representation  and  its  reconstruction  for  the 
compressed  signal  “chirp”.  The  trapezoidal  analyzing  function  g  appears  in  the  top  upper 
right.  To  its  left  are  the  functions  G  =  J2m=i  \Dam9\2  and  9-  The  middle  graph  displays  the 
reconstruction  of  Algorithm  3.4.3.  The  lower  most  graph  displays  the  sampling  set  T.  At  the 
top  of  the  bottom  graph  is  the  input  signal  and  to  its  lower  right  the  values  of  the  reconstruction 
parameters. 
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Figure  6.107:  Reconstruction  of  compressed  “chirp”,  (bc,br)  =  (1,1.2). 


Above,  the  signal  “chirp”and  its  reconstructed  versions  in  both  time  and  frequency  are 
displayed.  Below,  the  associated  coefficient  distribution  function  Achirp  is  plotted  in  the  top 
graph  while  the  lower  graph  shows  the  max  and  min  value  of  the  wavelet  filter  bank  response 
as  a  function  of  the  channel  number. 
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Figure  6.108:  Cdf  and  minimax  curves  for  compressed  “chirp”,  ( bc,br )  =  (1, 1.2). 
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Figure  6.109:  PE  wavelet  representation  for  “harmonic”,  (bc,br)  =  (1, 1.2). 

Shown  above  is  the  wavelet  positive  extrema  representation  and  its  reconstruction  for  the 
compressed  signal  “harmonic”.  The  trapezoidal  analyzing  function  g  appears  in  the  top  upper 
right.  To  its  left  are  the  functions  G  =  X3m=i  |-Dsm5|2  and  g.  The  middle  graph  displays  the 
reconstruction  of  Algorithm  3.4.3.  The  lower  most  graph  displays  the  sampling  set  T.  At  the 
top  of  the  bottom  graph  is  the  input  signal  and  to  its  lower  right  the  values  of  the  reconstruction 
parameters. 
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Figure  6.110:  Reconstruction  of  compressed  “harmonic”,  (bc,br)  =  (1,1.2). 


Above,  the  signal  “harmonic”  and  its  reconstructed  versions  in  both  time  and  frequency  are 
displayed.  Below,  the  associated  coefficient  distribution  function  Aharmonic  is  plotted  in  the  top 
graph  while  the  lower  graph  shows  the  max  and  min  value  of  the  wavelet  filter  bank  response 
as  a  function  of  the  channel  number. 


Figure  6.111:  Cdf  and  minimax  curves  for  compressed  “harmonic”,  ( bc,br )  =  (1, 1.2). 
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Figure  6.112:  PE  wavelet  representation  for  “water”,  ( bc,br )  =  (4,9.6). 


Shown  above  is  the  wavelet  positive  extrema  representation  and  its  reconstruction  for  the 
compressed  signal  “water”.  The  trapezoidal  analyzing  function  g  appears  in  the  top  upper 
right.  To  its  left  are  the  functions  G  —  l^sm?|2  ancl  9-  The  middle  graph  displays  the 

reconstruction  of  Algorithm  3.4.3.  The  lower  most  graph  displays  the  sampling  set  F.  At  the 
top  of  the  bottom  graph  is  the  input  signal  and  to  its  lower  right  the  values  of  the  reconstruction 
parameters. 
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Figure  6.113:  Reconstruction  of  compressed  “water”,  ( bc,br )  =  (4,9.6). 


Above,  the  signal  “water”and  its  reconstructed  versions  in  both  time  and  frequency  are 
displayed.  Below,  the  associated  coefficient  distribution  function  Awater  is  plotted  in  the  top 
graph  while  the  lower  graph  shows  the  max  and  min  value  of  the  wavelet  filter  bank  response 
as  a  function  of  the  channel  number. 
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Figure  6.114:  Cdf  and  minimax  curves  for  compressed  “water”,  ( bc,br )  —  (4,9.6). 
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Figure  6.115:  PE  wavelet  representation  for  “water”,  ( bc,br )  =  (2,9.6). 


Shown  above  is  the  wavelet  positive  extrema  representation  and  its  reconstruction  for  the 
compressed  signal  “water”.  The  trapezoidal  analyzing  function  g  appears  in  the  top  upper 
right.  To  its  left  are  the  functions  G  -  £m=i  \DSmg\2  and  g.  The  middle  graph  displays  the 
reconstruction  of  Algorithm  3.4.3.  The  lower  most  graph  displays  the  sampling  set  P.  At  the 
top  of  the  bottom  graph  is  the  input  signal  and  to  its  lower  right  the  values  of  the  reconstruction 
parameters. 
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Figure  6.116:  Reconstruction  of  compressed  “water”,  ( bc,br )  =  (2,9.6). 


Above,  the  signal  “water”  and  its  reconstructed  versions  in  both  time  and  frequency  are 
displayed.  Below,  the  associated  coefficient  distribution  function  AWilcr  is  plotted  in  the  top 
graph  while  the  lower  graph  shows  the  max  and  min  value  of  the  wavelet  filter  bank  response 
as  a  function  of  the  channel  number. 
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Figure  6.117:  Cdf  and  minimax  curves  for  compressed  “water”,  ( bc,br )  =  (2,9.6). 
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Figure  6.118:  PE  wavelet  representation  for  “water”,  ( bc,br )  =  (1,9.6). 


Shown  above  is  the  wavelet  positive  extrema  representation  and  its  reconstruction  for  the 
compressed  signal  “water”.  The  trapezoidal  analyzing  function  g  appears  in  the  top  upper 
right.  To  its  left  are  the  functions  G  =  J2m= l  \Dsm9\2  and  g-  The  middle  graph  displays  the 
reconstruction  of  Algorithm  3.4.3.  The  lower  most  graph  displays  the  sampling  set  F.  At  the 
top  of  the  bottom  graph  is  the  input  signal  and  to  its  lower  right  the  values  of  the  reconstruction 
parameters. 
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Figure  6.119:  Reconstruction  of  compressed  “water”,  ( bc,br )  =  (1,9.6). 


Above,  the  signal  “water”  and  its  reconstructed  versions  in  both  time  and  frequency  are 
displayed.  Below,  the  associated  coefficient  distribution  function  Awlter  is  plotted  in  the  top 
graph  while  the  lower  graph  shows  the  max  and  min  value  of  the  wavelet  filter  bank  response 
as  a  function  of  the  channel  number. 
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Figure  6.120:  Cdf  and  minimax  curves  for  compressed  “water”,  ( bc,br )  =  (1,9.6). 
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Figure  6.121:  PE  wavelet  representation  for  “water”,  ( bc,br )  =  (4,4.8). 


Shown  above  is  the  wavelet  positive  extrema  representation  and  its  reconstruction  for  the 
compressed  signal  “water”.  The  trapezoidal  analyzing  function  g  appears  in  the  top  upper 
right.  To  its  left  are  the  functions  G  —  Y^m=i  \DSrng\2  and  g.  The  middle  graph  displays  the 
reconstruction  of  Algorithm  3.4.3.  The  lower  most  graph  displays  the  sampling  set  P.  At  the 
top  of  the  bottom  graph  is  the  input  signal  and  to  its  lower  right  the  values  of  the  reconstruction 
parameters. 


177 


0  100  200  300  0  2  4  6  8 


Time  Reconstruction  Modulus  Frequency  Reconstruction 

Figure  6.122:  Reconstruction  of  compressed  “water”,  ( bc,bT )  =  (4,4.8). 


Above,  the  signal  “water”and  its  reconstructed  versions  in  both  time  and  frequency  are 
displayed.  Below,  the  associated  coefficient  distribution  function  Awatcr  is  plotted  in  the  top 
graph  while  the  lower  graph  shows  the  max  and  min  value  of  the  wavelet  filter  bank  response 
as  a  function  of  the  channel  number. 
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Figure  6.123:  Cdf  and  minimax  curves  for  compressed  “water”,  ( bc,br )  =  (4,4.8). 
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Figure  6.124:  PE  wavelet  representation  for  “water”,  ( bc,bT )  =  (2,4.8). 


Shown  above  is  the  wavelet  positive  extrema  representation  and  its  reconstruction  for  the 
compressed  signal  “water”.  The  trapezoidal  analyzing  function  g  appears  in  the  top  upper 
right.  To  its  left  are  the  functions  G  =  ]Cm=i  | DSmg\2  and  g.  The  middle  graph  displays  the 
reconstruction  of  Algorithm  3.4.3.  The  lower  most  graph  displays  the  sampling  set  F.  At  the 
top  of  the  bottom  graph  is  the  input  signal  and  to  its  lower  right  the  values  of  the  reconstruction 
parameters. 


179 


0  100  200  300  0  2  4  6  8 

Time  Reconstruction  Modulus  Frequency  Reconstruction 

Figure  6.125:  Reconstruction  of  compressed  “water”,  ( bc,br )  =  (2,4.8). 

Above,  the  signal  “water”  and  its  reconstructed  versions  in  both  time  and  frequency  are 
displayed.  Below,  the  associated  coefficient  distribution  function  Awater  is  plotted  in  the  top 
graph  while  the  lower  graph  shows  the  max  and  min  value  of  the  wavelet  filter  bank  response 
as  a  function  of  the  channel  number. 
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Figure  6.126:  Cdf  and  minimax  curves  for  compressed  “water”,  (bc,br)  —  (2,4.8). 
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Figure  6.127:  PE  wavelet  representation  for  “water”,  ( bc,br )  =  (1,4.8). 


Shown  above  is  the  wavelet  positive  extrema  representation  and  its  reconstruction  for  the 
compressed  signal  “water”.  The  trapezoidal  analyzing  function  g  appears  in  the  top  upper 
right.  To  its  left  are  the  functions  G  =  J2m=i  |-Dsm?|2  and  g.  The  middle  graph  displays  the 
reconstruction  of  Algorithm  3.4.3.  The  lower  most  graph  displays  the  sampling  set  F.  At  the 
top  of  the  bottom  graph  is  the  input  signal  and  to  its  lower  right  the  values  of  the  reconstruction 
parameters. 
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Figure  6.128:  Reconstruction  of  compressed  “water”,  ( bc,bT )  =  (1,4.8). 


Above,  the  signal  “water”  and  its  reconstructed  versions  in  both  time  and  frequency  are 
displayed.  Below,  the  associated  coefficient  distribution  function  Awiter  is  plotted  in  the  top 
graph  while  the  lower  graph  shows  the  max  and  min  value  of  the  wavelet  filter  bank  response 
as  a  function  of  the  channel  number. 
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Figure  6.129:  Cdf  and  minimax  curves  for  compressed  “water”,  ( bc,br )  =  (1,4.8). 
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Figure  6.130:  PE  wavelet  representation  for  “water”,  (bc,br)  =  (4,2.4). 


Shown  above  is  the  wavelet  positive  extrema  representation  and  its  reconstruction  for  the 
compressed  signal  “water”.  The  trapezoidal  analyzing  function  g  appears  in  the  top  upper 
right.  To  its  left  are  the  functions  G  =  J2m=i  \DSrng\2  and  g.  The  middle  graph  displays  the 
reconstruction  of  Algorithm  3.4.3.  The  lower  most  graph  displays  the  sampling  set  T.  At  the 
top  of  the  bottom  graph  is  the  input  signal  and  to  its  lower  right  the  values  of  the  reconstruction 
parameters. 
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Figure  6.131:  Reconstruction  of  compressed 
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water”,  ( bc,bT )  =  (4,2.4). 


Above,  the  signal  “water”and  its  reconstructed  versions  in  both  time  and  frequency  are 
displayed.  Below,  the  associated  coefficient  distribution  function  Awatei  is  plotted  in  the  top 
graph  while  the  lower  graph  shows  the  max  and  min  value  of  the  wavelet  filter  bank  response 
as  a  function  of  the  channel  number. 
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Figure  6.132:  Cdf  and  minimax  curves  for  compressed  “water”,  ( bc,br )  =  (4,2.4). 
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Figure  6.133:  PE  wavelet  representation  for  “water”,  ( bc,bT )  =  (2,2.4). 

Shown  above  is  the  wavelet  positive  extrema  representation  and  its  reconstruction  for  the 
compressed  signal  “water”.  The  trapezoidal  analyzing  function  g  appears  in  the  top  upper 
right.  To  its  left  are  the  functions  G  =  1  \Dsmd\2  and  g.  The  middle  graph  displays  the 

reconstruction  of  Algorithm  3.4.3.  The  lower  most  graph  displays  the  sampling  set  F.  At  the 
top  of  the  bottom  graph  is  the  input  signal  and  to  its  lower  right  the  values  of  the  reconstruction 
parameters. 
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Figure  6.134:  Reconstruction  of  compressed  “water”,  ( bc,br )  =  (2,2.4). 


Above,  the  signal  “water” and  its  reconstructed  versions  in  both  time  and  frequency  are 
displayed.  Below,  the  associated  coefficient  distribution  function  AWiter  is  plotted  in  the  top 
graph  while  the  lower  graph  shows  the  max  and  min  value  of  the  wavelet  filter  bank  response 
as  a  function  of  the  channel  number. 


Figure  6.135:  Cdf  and  minimax  curves  for  compressed  “water”,  ( bc,br )  =  (2,2.4). 
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Figure  6.136:  PE  wavelet  representation  for  “water”,  ( bc,br )  =  (1,2.4). 


Shown  above  is  the  wavelet  positive  extrema  representation  and  its  reconstruction  for  the 
compressed  signal  “water”.  The  trapezoidal  analyzing  function  g  appears  in  the  top  upper 
right.  To  its  left  are  the  functions  G  =  J2m= l  | DSmg\2  and  g.  The  middle  graph  displays  the 
reconstruction  of  Algorithm  3.4.3.  The  lower  most  graph  displays  the  sampling  set  T.  At  the 
top  of  the  bottom  graph  is  the  input  signal  and  to  its  lower  right  the  values  of  the  reconstruction 
parameters. 
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Figure  6.137:  Reconstruction  of  compressed  “water”,  ( bc,br )  =  (1,2.4). 


Above,  the  signal  “water”  and  its  reconstructed  versions  in  both  time  and  frequency  are 
displayed.  Below,  the  associated  coefficient  distribution  function  AWiter  is  plotted  in  the  top 
graph  while  the  lower  graph  shows  the  max  and  min  value  of  the  wavelet  filter  bank  response 
as  a  function  of  the  channel  number. 
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Figure  6.138:  Cdf  and  minimax  curves  for  compressed  “water”,  ( bc,br )  =  (1,2.4). 
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6.4  Discussion 


In  this  chapter  we  have  presented  numerous  examples  of  discrete  signal  representations. 
Examples  have  been  computed  and  displayed  for  cases  from  uniform  sampling  representations 
in  Paley- Wiener  spaces  to  positive  extrema  wavelet  representations  in  X2(1R)  or  More¬ 

over,  we  have  applied  the  notions  of  local  frames  and  frame  coherence  to  the  applications  of 
compression  and  noise  suppression. 

6.4.1  Paley  Wiener  Sampling 

We  have  provided  examples  which  illustrate  the  reconstruction  of  bandlimited  signals  from 
their  irregular  samples  using  Algorithm  3.4.3. 

A  sufficient  condition  given  by  the  Duffin-Schaeffer  Theorem  4.2.4  for  the  reconstruction 
of  a  bandlimited  signal  from  its  irregular  samples  is  that  the  sample  set  be  dense  enough, 
i.e.,  the  uniform  density  of  the  sampling  set  be  larger  than  twice  the  bandlimit.  For  signals 
with  complex  time-frequency  behavior,  e.g.  the  signal  “packet”  or  “chirp”,  the  Duffin-Schaeffer 
condition  may  lead  to  intuitively  unsatisfactory  sampling  sets.  For  example  for  the  “packet” 
signal  we  are  required  to  sample  at  a  density  consistent  with  the  highest  frequency  sine  packet 
in  the  signal.  For  the  other  lower  frequency  sine  packets  this  density  is  clearly  excessive.  Hence, 
for  Paley- Wiener  sampling  there  is  no  mechanism  for  time  localization.  In  other  words  it  is 
necessary  to  sample  densely  over  the  entire  duration  of  the  signal  even  if  the  high  frequency 
components  of  the  signal  are  well  localized  in  time.  This  situation  naturally  suggests  a  joint 
time-frequency  analysis  such  as  afforded  by  the  wavelet  transform. 

6.4.2  PE  wavelet  Representation 

Some  of  the  general  properties  of  the  PE  wavelet  representations  with  trapezoidal  analyzing 
function  which  have  been  observed  here  are  listed  below.  Recall  that  the  trapezoidal  analyzing 
has  very  good  localization  in  both  time  and  frequency. 

Fast  Convergence  For  all  reconstructions  based  on  the  PE  wavelet  extrema  representation 
we  have  exhibited  only  the  first  iteration  of  Algorithm  3.4.3  with  relaxation  parameter  A  =  0.1. 
As  can  be  seen  from  the  previous  simulations  the  first  iteration  in  every  case  is  already  a  high 
quality,  though  multiplicatively  scaled  (by  A),  version  of  the  original.  It  has  been  experimentally 
observed  that  further  iterations  serve  only  to  scale  the  reconstruction  properly.  We  attribute 
this  situation  to  the  choice  of  well  localized  time-frequency  analyzing  function  g  =  </trip  and 
the  fact  that  the  function  G  of  Lemma  4.2.1  is  essentially  constant  over  the  frequency  range  of 
interest. 

Quantization  robustness  Reconstructions  via  Algorithm  3.4.3  exhibit  a  high  degree  of  ro¬ 
bustness  to  quantization  of  coefficients.  Quantization  of  the  coefficients  at  levels  as  low  as  one 
or  two  bits  per  coefficient  still  result  in  high  quality  approximations  of  the  original  signal.  This 
suggests  the  applicability  of  such  representations  to  data  and  speech  compression. 
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Embedded  truncated  representations  An  appealing  property  of  the  truncated  wavelet 
representation  of  a  signal  /  is  that  the  truncated  representations 

A S  =  sm)  •  fE gf*(tm,nism )  ^ 

form  a  decreasing  continuum  of  sets  with  respect  to  the  threshold  parameter  S.  In  other  words, 
if  <  S2  then  A$2  C  A$,.  Thus,  the  truncation  is  hierarchical  in  the  sense  that  representations 
with  small  information  content  A$2  are  embedded  in  ones  with  higher  information  content  A^ . 

Robust  truncation  It  has  been  experimentally  observed  that  (for  coherent  signals)  as  the  PE 
representation  is  truncated  through  the  application  of  smaller  thresholds  6  that  reconstructions 
based  on  initializing  Algorithm  3.4.3  with  the  data  A$  degrade  in  a  robust  way.  Because  the 
truncated  representations  are  naturally  embedded  (see  above),  the  increase  of  the  threshold 
S  from  Ai  to  S2  (Si  <  S2)  corresponds  to  the  removal  of  some  elements  of  A$1 .  This  in  turn 
corresponds  to  removing  the  least  significant  components  of  the  reconstruction  based  on  Agj. 
This  insures  that  a  small  change  in  6  will  not  lead  to  a  catastrophic  change  in  the  reconstruction. 

Time  Translation  Invariance  The  PE  wavelet  representation  is  translation  invariant  in 
the  sense  that  a  time  shifted  version  of  a  signal  will  yield  a  corresponding  time  shifted  PE 
wavelet  representation.  This  is  simply  because  the  extrema  points  of  a  function  are  translation 
invariant,  i.e.,  if  a  signal  is  time  shifted  one  unit  to  the  left  so  are  its  extrema.  Translation 
invariance  is  an  important  property  in  pattern  recognition  schemes  where  it  is  desirable  for  time 
shifted  versions  of  signals  to  yield  identical  representations. 

6.4.2. 1  Noise  Suppression 

From  the  numerical  experiments  we  have  observed  the  ability  to  reject  wide  band  random 
noise  in  signals  with  coherent  time-frequency  structure.  The  basic  approach  to  noise  suppression 
has  been  founded  on  the  notion  of  frame  coherence,  viz.  Section  3.5.1,  and  localization  through 
application  of  a  thresholding  operation  on  the  (frame)  representation  of  a  noise  corrupted  signal. 

The  minimax  curves  in  the  simulations  illustrate  the  notion  of  coherence.  Recall  that  the 
minimax  curves  show  the  minimum  and  maximum  values  attained  by  the  output  of  a  specific 
filter  in  the  continuous  wavelet  representation  of  the  signal.  Comparing  the  minimax  curves  for 
a  specific  signal  it  can  be  seen  that  as  the  noise  level  is  increased  the  overall  shape  of  the  curve 
due  to  the  coherent  portion  is  retained  while  the  gap  between  the  minimum  and  maximum 
curves  is  increased  uniformly  over  all  channels  due  to  the  incoherent  portion.  This  is  because 
the  incoherent  portion  must  necessarily  spread  its  energy  throughout  the  entire  filter  bank.  For 
this  reason  we  should  choose  a  threshold  which  rejects  the  gap  area.  Clearly,  in  high  noise 
situations  this  procedure  will  break  down  since  the  non-coherent  portion  may  rise  to  a  level 
comparable  to  the  signal. 

6.4. 2. 2  Compression 

Examining  the  results  of  our  compression  experiment  we  can  make  the  following  observa¬ 
tions: 
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(i)  In  general,  reconstructions  are  ‘good’  for  all  values  of  6C=1,2  or  4.  We  use  the  term  ‘good’ 
in  the  sense  that  both  the  time  and  frequency  magnitude  reconstructions  are  judged  to 
be  close  to  their  original  counterparts. 

(ii)  Frequency  magnitude  reconstructions  degrade  less  severely  than  time  reconstructions  as 
bc  varies  from  4  to  1  bit  per  coefficient. 

(iii)  Strong  frequency  components  (peaks)  are  replicated  faithfully. 

The  results  of  the  experiment  also  indicate  that  the  PE  representation  is  highly  robust 
to  quantization  effects.  Allocating  just  one  or  two  bits  per  coefficient  (6C  =  1,2)  still  allows 
for  good  quality  reconstructions.  It  is  here  that  we  see  one  benefit  of  non-orthogonal  highly 
redundant  systems,  viz.  Section  3.5.1.  All  of  these  observations  suggest  that  the  proposed  data 
compression  scheme  is  a  promising  one  for  speech. 

There  are  other  variables,  trade-offs,  and  issues  for  evaluating  the  compression  scheme.  We 
list  some  of  them  here. 

(i)  Besides  coefficient  quantization,  time  quantization  must  also  be  addressed.  Each  PE 
wavelet  coefficient  for  a  signal  /  is  a  sample  Wgf(sm,tm,n)  of  the  wavelet  transform  Wgf. 
Since  the  sequence  {fm,n}  is  signal  dependent  for  the  PE  representation,  the  reconstruction 
process  (receiver)  must  have  knowledge  of  these  values.  In  our  reconstructions  we  have 
assumed  complete  knowledge  of  the  sequence  {tm>n}. 

(ii)  Essentially,  we  have  circumvented  the  issue  of  windowing  of  the  signal  by  taking  the 
window  to  be  of  a  length  equal  to  the  duration  of  the  signal.  In  practice,  windowing  plays 
a  finer  role,  e.g.,  in  narrowband  speech  compression  systems  going  back  to  Dudley  (1939), 
speech  coders  divide  the  speech  signal  into  intervals  of  duration  10  to  25  ms. 

(iii)  The  dilation  parameter  ao  effectively  changes  the  frequency  support  of  each  filter  in  the 
filter  bank  {(Dakdg)}.  In  particular,  increasing  the  value  of  a0  decreases  the  bandwidth 
of  each  filter  in  the  bank.  A  decrease  in  bandwidth  necessarily  implies  that  a  particular 
filter  will  respond  to  a  smaller  band  in  frequency.  Thus,  keeping  /  the  same,  an  increase  in 
the  value  of  ao  will  cause  the  bands  of  activity  in  the  original  ao  representation  to  become 
compressed  along  the  5- axis  in  the  new  increased  ao  representation.  Thus,  with  6  fixed, 
larger  a0  will  lead  to  smaller  data  sets.  On  the  other  hand,  larger  values  of  ao  will  cause 
the  function  G  in  4.2.1  to  have  greater  variation  from  constant  value.  This  condition 
necessarily  implies  a  spread  between  possible  frame  bounds,  i.e.,  movement  away  from 
tightness.  This,  in  turn,  can  be  related  to  a  slowing  of  the  rate  of  convergence  of  the 
reconstruction  Algorithm  3.4.3. 

(iv)  An  interesting  inherent  feature  of  the  compression  scheme  and  representation  is  that,  in 
a  communication  setting,  all  of  the  available  bandwidth  can  be  used  for  the  transmission 
of  information  about  the  underlying  signal.  Suppose  a  fixed  information  transmission 
rate  limit,  e.g.,  the  bit  rate  constraint  bT.  Further,  suppose  that  a  threshold  of  S  =  0 
yields  a  finite  PE  wavelet  representation  Ao (/*)  from  which  it  is  possible  to  reconstruct 
perfectly  the  original  signal  /*.  Clearly,  the  cardinality  of  the  representation  Ao (/*) 
depends  on  the  information  content  of  the  underlying  signal  /*.  For  example,  if  /*  =  0 
then  card  As  =  0  for  all  values  of  6.  For  signals  with  low  enough  information  content, 
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i.e.,  bc  •  card  Ao  <  br  •  |/(/„)|,  the  information  rate  constraint  poses  no  problem.  For 
more  complex  signals,  though,  a  threshold  of  zero  will  not  suffice  to  meet  the  information 
constraint.  Suppose  we  have  such  a  signal.  In  this  case,  an  appropriate  threshold  must  be 
found  via  the  distribution  function  in  (5.3.3).  Since  the  thresholded  representations  are 
embedded  (see  above),  the  PE  wavelet  compression  scheme  can  be  viewed  as  a  method 
to  remove  the  least  significant  coefficients  from  the  set  Ao  to  meet  the  information  rate 
constraint.  Removing  the  least  significant  coefficients  insures  that  the  least  amount  of 
information  will  be  lost.  The  significance  of  this  is  that  (i)  the  PE  wavelet  compression 
scheme  yields  the  best  thresholded  representation  which  meets  the  information  constraint; 
and  (ii)  if  the  information  constraint  br  were  increased,  the  new  PE  wavelet  compressed 
representation  would  contain  the  old  PE  wavelet  compressed  representation.  This  is  a 
consequence  of  the  fact  that  the  information  constraint  br  can  be  related  directly  to  the 
threshold  6  by  the  relation 

bc  card  Ag/\I(f*)\  <  bT, 

and  that  card  A$  is  a  decreasing  function  of  S.  If  b\  <  b%  are  two  information  rate  con¬ 
straints,  the  two  corresponding  thresholded  representations  generated  by  the  PE  wavelet 
compression  scheme  will  be  A^  and  A s2,  and  £1  >  S2  so  that  Ag1  C  Ag2.  Consequently, 
over  different  signals  (or  pieces  of  signals)  it  is  always  possible  to  transmit  at  a  rate  up  to 
the  information  limit  br  by  adjusting  the  threshold  S. 

(v)  We  have  not  dealt  with  the  issue  of  the  choice  of  the  analyzing  function  aside  from 
designing  it  to  have  good  time  and  frequency  localization.  For  the  compression  of  speech 
data  it  seems  reasonable  to  suspect  that  analyzing  functions  which  come  from  a  model  of 
the  auditory  system  will  provide  better  results.  This  issue  has  been  studied  by  Benedetto 
and  Teolis  [BT92a],  [BT92b]. 
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Chapter  7 

Conclusion 


We  conclude  this  thesis  with  a  brief  summary  and  list  of  directions  for  future  research. 
Highlighted  in  the  summary  is  a  list  of  some  of  the  contributions  made  by  this  research. 

7.1  Summary 

In  this  thesis  we  have  presented  a  constructive  theory  for  the  discrete  representation  of 
analog  signals  from  infinite  dimensional  Hilbert  spaces.  The  approach  to  discrete  representation 
which  we  have  taken  has  its  fundamental  roots  in  the  theory  of  frames  and  may  be  analyzed  in 
terms  of  irregular  sampling  in  certain  reproducing  kernel  Hilbert  spaces.  In  its  most  general  form 
the  discrete  representation  of  a  signal  is  the  irregular  samples  of  a  signal  dependent  function 
whose  domain  is  a  group,  e.g.,  the  real  line  under  addition,  or  the  affine  group.  Special  cases 
of  this  general  transform  are  the  Gabor  and  Wavelet  transform. 

With  the  perspective  of  digital  implementation,  we  have  developed  algorithms  for  the  recon¬ 
struction  of  signals  from  their  discrete  representations.  In  addition,  we  have  studied  the  effects 
of  quantization  and  perturbations  in  the  representation  domain  with  respect  to  reconstructions. 

We  have  introduced  the  notion  of  frame  localization  with  respect  to  a  specific  signal.  Lo¬ 
calization  leads  to  finite  discrete  representations  which  efficiently  capture  the  essential  charac¬ 
teristics  of  a  signal. 

Further,  we  have  implemented  the  theory  and  provided  numerous  examples  of  representa¬ 
tions  and  their  reconstructions.  For  the  particular  case  of  the  positive  wavelet  extrema  rep¬ 
resentation  we  have  implemented  schemes  for  noise  suppression  and  compression.  Results  of 
these  applications  can  be  viewed  in  Chapter  6. 

In  the  following  list  we  indicate  some  of  the  main  contributions  of  this  research: 

•  A  general  reproducing  kernel  Hilbert  space  frame  representation  approach  to  the  repre¬ 
sentation  of  signals  from  infinite  dimensional  Hilbert  spaces. 

•  The  development  of  iterative  algorithms  for  the  reconstruction  of  a  signal  from  its  discrete 
representation  which  is  cast  in  terms  of  the  frame  correlation  operator,  viz.  Section  3.3. 

•  Identification  of  the  role  of  the  pseudo-inverse  frame  correlation  operator  in  reconstruction 
from  perturbed  representations  and  its  relation  to  the  underlying  frame  bounds  in  infinite 
dimensional  spaces,  viz.  Theorem  3.3.6. 
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•  The  development  of  an  iterative  algorithm  for  the  reconstruction  of  a  signal  from  its 
perturbed  representation,  e.g.,  from  quantization  of  the  representation. 

•  Introduction  of  the  notion  of  frame  localization  through  truncation  and  signal  dependent 
sampling  sets. 

•  Introduction  of  the  positive  extrema  wavelet  representation  for  the  efficient  representation 
of  signals  with  complex  time-frequency  behavior. 

•  A  discrete  representation  method  for  the  suppression  of  noise  based  on  the  notion  of  frame 
coherence,  viz.  Section  6.3.1. 

•  A  discrete  representation  method  for  the  compression  of  signals,  viz.  Section  6.3.2. 

7.2  Future  Directions 

Future  research  efforts  may  be  thought  of  in  two  broad  areas.  First,  there  are  extensions  of 
the  theory  and  second,  there  is  the  investigation  and  development  of  further  applications. 

7.2.1  Theory 

Some  potential  theoretical  directions  on  which  future  research  could  focus  are  listed  below. 

Extension  of  the  theory  to  higher  dimensions  In  particular  a  two  dimensional  extension 
of  the  theory  would  find  immediate  applicability  in  image  processing. 

Incorporation  of  auxiliary  information  In  many  applications  there  is  auxiliary  informa¬ 
tion  known  about  a  signal.  For  example,  the  signal  may  be  known  to  be  positive  in  certain 
intervals.  In  such  cases,  we  expect  that  signals  should  be  able  to  be  represented  more  efficiently 
with  side  information  than  without  it. 

Investigation  of  zero  crossing  sets  In  [Log77]  Logan  shows  that  certain  classes  of  (octave) 
bandlimited  functions  may  be  reconstructed  from  knowledge  of  only  their  zero  crossings  to 
within  a  multiplicative  constant;  however,  Logan’s  original  work  is  non-constructive  in  nature. 
The  analysis  of  zero  crossings  in  terms  of  the  frame  approach  we  have  taken  in  this  thesis  may 
lead  to  a  constructive  theory  of  reconstruction  from  zero  crossing  sets  for  octave  bandlimited 
functions.  Moreover,  extensions  to  larger  spaces  may  be  attacked  with  an  approach  similar  to 
that  of  Lemma  4.2.1. 

Analysis/Design  of  discrete  representations  There  are  many  free  parameters  present 
in  the  generic  discrete  representations  which  we  have  developed  here,  viz.  Chapter  4.  These 
free  parameters  include  the  group,  group  representation,  analyzing  function  g ,  and  discrete 
sampling  lattice  in  the  group.  In  many  cases  some  or  all  of  these  parameters  will  be  fixed 
independent  of  the  class  of  signals  of  interest.  For  a  given  problem,  however,  it  may  prove 
fruitful  to  tailor  some  of  these  parameters.  A  detailed  analysis  of  these  sorts  of  considerations 
could  lead  to  efficient  and  useful  representations  of  signals.  For  example,  the  positive  extrema 
wavelet  representation  illustrated  in  Chapter  6  is  based  on  signal  dependent  samplings  where 
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the  group,  group  representation,  and  analyzing  function  are  all  fixed  apriori.  There,  the  group 
and  group  representation  have  been  selected  to  yield  the  wavelet  transform.  In  this  case,  one 
direction  of  future  research  would  involve  the  investigation  of  the  design  of  analyzing  functions 
to  better  suit  the  signals  to  be  analyzed. 

7.2.2  Application 

In  this  thesis  we  have  explored  only  two  potential  applications  of  the  discretization  theory. 
Namely,  we  have  examined  applications  to  compression  and  noise  suppression.  There  are  a  vast 
number  of  other  areas  in  signal  and  speech  processing  for  which  discrete  representations  have 
applicability.  A  partial  list  of  some  of  these  is  given  below. 

Signal  Detection/Recognition  From  the  simulations  provided  in  Chapter  6  for  the  case  of 
the  positive  extrema  wavelet  representation,  there  is  an  indication  that  such  a  representation 
captures  the  time-frequency  content  of  the  signal.  Moreover,  we  have  seen  that  these  represen¬ 
tations  are  robust  to  noise  and  enjoy  the  property  of  translation  invariance.  These  observations 
indicate  that  such  a  representation  is  well  suited  for  detection'  or  recognition. 

Parameter  Estimation  For  signals  which  may  be  described  by  one  or  more  parameters  dis¬ 
crete  representations  offer  a  method  of  estimating  the  signal  parameter(s).  In  Chapter  6  we 
have  seen  that  for  the  case  of  the  positive  extrema  wavelet  representation  the  time-frequency 
evolution  of  a  signal  is  captured.  For  signals  whose  time-frequency  behavior  may  be  param¬ 
eterized,  e.g.  frequency  modulated  (FM)  signals,  the  positive  extrema  wavelet  representation 
should  provide  enough  information  to  estimate  the  modulation  parameters.  For  example  the 
parameter  a  in  the  chirp  signal  sin  (of2)  can  be  estimated  from  any  one  of  its  representations 
in  Section  6.2. 2.3. 

Synthesis/Resynthesis  We  have  heretofore  focused  on  the  process  of  representing  a  signal 
in  a  discrete  manner.  We  may  turn  the  situation  around  and  ask  if  it  is  possible  to  directly 
construct  discrete  representations  which  when  reconstructed  have  desirable  or  interesting  prop¬ 
erties.  Or  in  a  related  question  we  may  ask  the  following:  given  a  discrete  representation  of 
a  signal  may  that  representation  be  altered  to  achieve  some  desired  effect  in  the  signal  do¬ 
main?  Respectively,  these  two  tasks  are  called  synthesis  and  resynthesis.  Development  of  such 
techniques  may  prove  useful  in  music  for  the  synthesis/resynthesis  of  interesting  timbres. 

Time  Dilation  An  interesting  problem  in  sound  processing  is  the  speeding  up  or  slowing  down 
of  recorded  sound  while  preserving  the  frequency  support  of  the  signal.  A  possible  approach  to 
this  problem  is  to  develop  processing  techniques  in  the  representation  domain  which  perform  the 
requisite  speeding  or  slowing  of  signals.  For  instance,  for  the  positive  extrema  representations 
of  Chapter  6  a  simple  time  dilation  in  the  coefficient  domain  would  change  the  speed  in  signal 
domain  but  not  the  frequency  support. 
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Appendix  A 

Assorted  Items 


Proposition  A.l  If  K  :  Hi  >-+  H2  is  a  bounded  linear  operator  mapping  the  Hilbert  space  Hi 
into  the  closed  Hilbert  space  H2  then  the  adjoint  operator  K*  maps  H2  onto  Hi. 

Proof: 

We  first  prove  that  the  range  K*(H2)  is  closed.  Since  Hi  is  closed,  any  converging  sequence 
{j/n.}  Q  H2  converges  to  a  y  =  lim yn  £  Hi-  Through  application  of  K*  we  may  generate 
another  sequence  xn  =  K*yn  so  that  {xn}  C  K*(H2)-  Because  K*  is  bounded  and  linear  and 
therefore  continuous  we  have  that  xn  converges  and 

z  =  lim  xn  =  lim  K*yn  =  yn)  =  K*y 

where  the  exchange  of  limit  is  justified  since  K*  is  continuous.  We  conclude  that  x  £  K*(Hi) 
because  y  £  7f2.  Thus  we  have  proven  that  K*{H.2)  is  closed. 

Using  the  basic  fact  from  operator  theory  that 

K*(H2)  =  (kerfif)1, 

and  that  K  is  1-1,  i.e.  ker/if  =  {0},  we  have 

K*(H2)  =  K*(H2 )  =  {0}-1  =  Hi. 


This  proves  that  K*  is  onto  Hi . 


Corollary  A. 2  If  in  addition  to  the  hypotheses  of  A.l  I(  is  self  adjoint  (H  =  Hi  —  H2  is 
closed)  then  K*  is  both  surjective  and  injective,  i.e.  bijective,  and  therefore  invertible  on  K(H). 
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Appendix  B 

Group  Representation 


In  this  appendix  we  detail  the  computations  which  confirm  that  11//  and  11^  are  group 
representations  of  Gh  and  Ga  respectively  on  L2(IR)  with  the  group  operations  indicated  in 
Section  2.3. 


B.l  Weyl-Heisenberg 

The  Weyl-Heisenberg  group  Gh  is  T  X  It  X  1R  and  the  representation  we  are  using  is 
HgCM.t)  =  ze^Tf.  For  this  to  be  a  valid  representation  we  must  have  II(a;)n(T/)  =  ll(a:  •  y )  for 
all  x,  y  €  Gh •  Note  the  operator  identity 

_  .  __  —2Triab „ 

where  a  and  b  are  real  numbers.  Let  Xi  =  (^i,<i,7i)  and  x2  =  ( z2,t2,~f2 )  be  arbitrary  members 
of  Gh-  Then 


nz/(zi)nz/(:r2)  =  z-lellTtlz2el2Tt7 

=  Ziz2e~2m'y2tle11  el2  rtl  rt2 
=  ziZ2e~2m'l2he^1+l2Th+t2. 

Thus,  the  group  operation  must  be 

0Mi,7i)  '  (z2, t2,  j2)  =  (z1z2e~27r,y2tl,ti  +t2, 7i  +72). 

With  this  operation  it  is  easy  to  check  that  the  identity  is  e  =  (1,0,0),  i.e.  Vx  E  Gh,  x  •  e  = 
e  •  x  —  x.  If  x2  =  xj"1  is  the  inverse  of  we  must  have  e  —  x\  ■  x2  ox 

(1,0,0)  =  (z1z2e~2*,'ntl  +  t2, 71  +  72). 

By  direct  substitution  it  can  be  seen  that  this  is  satisfied  by 

x-' =  {z-le~2^t\~tu-ll). 
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B.2  Affine 


The  affine  group  Qa  is  1R  X  EL+  and  the  representation  we  are  using  is  s )  =  rtDs.  For 
this  to  be  a  valid  representation  we  must  have  n(x)II(2/)  =  II(a:  •  y)  for  all  x,y  £  Qjj-  Note  the 
operator  identity 

DaTb  =  Ta—lbD a, 

where  a  >  0  and  b  are  real  numbers.  Let  xi  =  and  x2  =  (s2,t2)  be  arbitrary  members 

of  Qa-  Then 

n^(*x)IIfl-(a:2)  =  TtlDSlThDS2 

~  Ttl  TS~1 12  Ds2 

=  Tt1+S~1t2Ds  1*2 

Thus,  the  group  operation  must  be 

(si,fl)  •( S2,t2 )  =  (ti  +Si1t2,S  1S2)- 

With  this  operation  it  is  easy  to  check  that  the  identity  is  e  =  (0, 1),  i.e.  Va;  £  Qjj,  x-e  =  e-x  =  x. 
If  x2  =  is  the  inverse  of  Xi  we  must  have  e  =  x\  •  x2  or 

(0,1)  =  (h  +  Silt2,SiS2). 

By  direct  substitution  it  can  be  seen  that  this  is  satisfied  by 

xTl  = 
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Appendix  C 

Generalized  Inverses 


This  appendix  provides  some  of  the  basic  concepts  associated  with  generalized  inverses 
of  linear  operators  which  are  pertinent  to  frame  representations  and  reconstructions.  For  a 
detailed  treatment  of  the  subject  the  reader  is  referred  to  [Gro77]. 

There  are  several  equivalent  definitions  of  a  “generalized  inverse”  of  a  linear  operator.  We 
adopt  the  definition  attributed  to  Penrose  [Pen55]  and  use  the  terms  “generalized  inverse”  and 
“pseudo  inverse”  interchangeably. 


Definition  C.l[Gro77,  Definition  (P)]  If  A  G  f?(7fi,7f2)  ^as  dosed  range,  then  A t  is  the 
unique  operator  in  B(H2,H\)  satisfying 

(1)  A  At  is  self  adjoint, 

(2)  At  A  is  self  adjoint, 

(3)  AAt  A  =  A,  and 

(4)  At  AAt  =  At. 


Fact  C.2  Let  A  G  !?(?fi,7f2)  have  closed  range.  The  mapping  At  :  7f2  Hi  defined  by 

A=  ^711, -.4*11 
y 


is  the  generalized  inverse  of  A. 
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